arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.05025cs.CV
收藏

RAMEN: 面向地球观测的分辨率可调多模态编码器

RAMEN: Resolution-Adjustable Multimodal Encoder for Earth Observation

Institut National des Sciences de l'Univers (INSU), France(法国国家科学研究院);CNRS, France(法国国家科学研究中心)

Nicolas Houdré, Diego Marcos, Hugo Riffaud de Turckheim, Dino Ienco, Laurent Wendling, Camille Kurtz, Sylvain Lobry

AI总结:

提出RAMEN,一种传感器无关的分辨率可调多模态编码器,通过将分辨率作为可控参数,在统一潜空间中实现多模态地球观测数据的连贯分析,并在PANGAEA基准上优于现有模型。

摘要:

地球观测(EO)数据涵盖广泛的空间、光谱和时间分辨率,从高分辨率光学图像到低分辨率多光谱产品或雷达时间序列。虽然最近的基座模型改进了多模态集成以学习有意义的表示,但它们通常期望固定的输入分辨率或基于传感器特定的编码器,限制了跨异构EO模态的泛化。为克服这些限制,我们引入了RAMEN,一种分辨率可调的多模态编码器,以完全传感器无关的方式学习跨EO数据的共享视觉表示。RAMEN将模态、空间和时间分辨率视为关键输入数据特征,从而在统一潜空间内实现跨模态的连贯分析。其主要方法贡献是将空间分辨率定义为可控输出参数,使用户在推理时能够直接控制所需的细节水平,并允许在空间精度和计算成本之间进行显式权衡。我们训练了一个统一的Transformer编码器,用于重构来自不同来源的掩蔽多模态EO数据,确保跨传感器和分辨率的泛化。预训练后,RAMEN有效地迁移到已知和未见过的传感器配置,并在社区标准的PANGAEA基准上优于更大的最先进模型,该基准包含多种多传感器和多分辨率下游任务。我们的代码和预训练模型可在以下网址获取:https://this URL。

英文摘要:

Earth observation (EO) data spans a wide range of spatial, spectral, and temporal resolutions, from high-resolution optical imagery to low resolution multispectral products or radar time series. While recent foundation models have improved multimodal integration for learning meaningful representations, they often expect fixed input resolutions or are based on sensor-specific encoders limiting generalization across heterogeneous EO modalities. To overcome these limitations we introduce RAMEN, a resolution-adjustable multimodal encoder that learns a shared visual representation across EO data in a fully sensor-agnostic manner. RAMEN treats the modality and spatial and temporal resolutions as key input data features, enabling coherent analysis across modalities within a unified latent space. Its main methodological contribution is to define spatial resolution as a controllable output parameter, giving users direct control over the desired level of detail at inference and allowing explicit trade-offs between spatial precision and computational cost. We train a single, unified transformer encoder reconstructing masked multimodal EO data drawn from diverse sources, ensuring generalization across sensors and resolutions. Once pretrained, RAMEN transfers effectively to both known and unseen sensor configurations and outperforms larger state-of-the-art models on the community-standard PANGAEA benchmark, containing various multi-sensor and multi-resolution downstream tasks. Our code and pretrained model are available at https://github.com/nicolashoudre/RAMEN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑