arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.11706cs.LGcs.CV
收藏

面向情境的多模态表示学习用于时空明确的环境建模

Context-Aware Multimodal Representation Learning for Spatio-Temporally Explicit Environmental Modelling

Environmental Data Science and Remote Sensing Group(环境数据科学与遥感小组);Institute for Earth System Science and Remote Sensing(地球系统科学与遥感研究所);Leipzig University(莱比锡大学);German Centre for Integrative Biodiversity Research (iDiv)(整合生物多样性研究德国中心(iDiv))

Julia Peters, Karin Mora, Miguel D. Mahecha, Chaonan Ji, David Montero, Clemens Mosig, Guido Kraemer

AI总结:

本文提出一种面向情境的多模态表示学习框架,整合不同地球观测模态以高时空分辨率建模环境,提升生态分析的精度与效率。

摘要:

地球观测(EO)基础模型已作为一种有效方法,从各种遥感传感器中推导出地球系统的潜在表示。这些模型生成的嵌入可以作为分析准备的数据集,使在不进行大量传感器特定预处理的情况下建模生态系统动态成为可能。然而,现有模型通常在固定的空或时间尺度上运行,限制了其在需要精细空间细节和高时间保真的生态分析中的应用。为克服这些限制,我们提出了一种表示学习框架,将不同的EO模态整合到一个统一的特征空间中,以高时空分辨率进行。我们使用Sentinel-1和Sentinel-2数据作为代表性模态来介绍该框架。我们的方法在原生10米分辨率和无云Sentinel-2采集的时频上生成潜在空间。每个传感器首先被独立建模以捕捉其传感器特定的特征。然后将它们的表示结合到一个共享模型中。这种两阶段设计使模态特定的优化和易于扩展到新传感器成为可能,保留了预训练的编码器,同时仅重新训练融合层。这使模型能够捕捉互补的遥感数据,并在空间和时间上保持一致性。定性分析表明,所学的嵌入在异质景观中表现出高空间和语义一致性。定量评估在建模总初级生产力时揭示,它们编码了生态上有意义的模式,并保留了足够的时间保真度以支持细粒度分析。总体而言,所提出的框架为需要不同空间和时间分辨率的环境应用提供了一种灵活、分析准备的表示学习方法。

英文摘要:

Earth observation (EO) foundation models have emerged as an effective approach to derive latent representations of the Earth system from various remote sensing sensors. These models produce embeddings that can be used as analysis-ready datasets, enabling the modelling of ecosystem dynamics without extensive sensor-specific preprocessing. However, existing models typically operate at fixed spatial or temporal scales, limiting their use for ecological analyses that require both fine spatial detail and high temporal fidelity. To overcome these limitations, we propose a representation learning framework that integrates different EO modalities into a unified feature space at high spatio-temporal resolution. We introduce the framework using Sentinel-1 and Sentinel-2 data as representative modalities. Our approach produces a latent space at native 10 m resolution and the temporal frequency of cloud-free Sentinel-2 acquisitions. Each sensor is first modeled independently to capture its sensor-specific characteristics. Their representations are then combined into a shared model. This two-stage design enables modality-specific optimisation and easy extension to new sensors, retaining pretrained encoders while retraining only fusion layers. This enables the model to capture complementary remote sensing data and to preserve coherence across space and time. Qualitative analyses reveal that the learned embeddings exhibit high spatial and semantic consistency across heterogeneous landscapes. Quantitative evaluation in modelling Gross Primary Production reveals that they encode ecologically meaningful patterns and retain sufficient temporal fidelity to support fine-scale analyses. Overall, the proposed framework provides a flexible, analysis-ready representation learning approach for environmental applications requiring diverse spatial and temporal resolutions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑