← 研踪 PaperTrailCVF · CVPR 2026:cvf:bc848be20d4a39697e1c6184fa32298e · 记录 #1
Beyond What's Shared: Recovering Lost Unique Information from Intermediate Layers to Boost Multimodal Geo-Foundation Models
来源日期（仅到年份） 2026
来源记录日期 2026
雷达观测 2026-11-05 04:11
摘要
Learning general-purpose representations of geographic locations has become essential to geospatial tasks such as population estimation and environmental monitoring. To obtain such representations, multimodal geo-foundation models often use contrastive learning (CL) to align satellite imagery with geo-coordinates, implicitly assuming that cross-modal (shared) information suffices for downstream tasks. However, given the breadth of tasks, task-relevant information may lie beyond the shared space, so retaining modality-specific (unique) features can improve task performance. Prior methods retain unique information through extra training objectives or external models, increasing training complexity. Motivated by the conventional wisdom that earlier layers capture general input features while later layers become task-specific, we hypothesize that intermediate layers in CL models retain more modality-specific structure than the alignment-optimized final layer. Through a trifecta layerwise analysis of modality gap, representation similarity, and mutual information, we validate this trend and find that fusing intermediate (more unique) and final (more shared) representations yields consistent gains on diverse geospatial tasks. Our findings reveal underutilized information diversity in CL models and show that simple layerwise fusion is an efficient path to richer geo-embeddings.
论文发现出处
这些页面提供论文线索，不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化，不是作者修订号。来源日期仅到月份或年份时，按该月或该年首日存档。
记录 #1 · 2026 · 最新观测版本
Beyond What's Shared: Recovering Lost Unique Information from Intermediate Layers to Boost Multimodal Geo-Foundation Models
Learning general-purpose representations of geographic locations has become essential to geospatial tasks such as population estimation and environmental monitoring. To obtain such representations, multimodal geo-foundation models often use contrastive learning (CL) to align satellite imagery with geo-coordinates, implicitly assuming that cross-modal (shared) information suffices for downstream tasks. However, given the breadth of tasks, task-relevant information may lie beyond the shared space, so retaining modality-specific (unique) features can improve task performance. Prior methods retain unique information through extra training objectives or external models, increasing training complexity. Motivated by the conventional wisdom that earlier layers capture general input features while later layers become task-specific, we hypothesize that intermediate layers in CL models retain more modality-specific structure than the alignment-optimized final layer. Through a trifecta layerwise analysis of modality gap, representation similarity, and mutual information, we validate this trend and find that fusing intermediate (more unique) and final (more shared) representations yields consistent gains on diverse geospatial tasks. Our findings reveal underutilized information diversity in CL models and show that simple layerwise fusion is an efficient path to richer geo-embeddings.
查看此版本解读 →查看此版本原文 ↗