Galileo：学习多种遥感模态的全局与局部特征
Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
AI总结：
Galileo 提出一种多模态 transformer 与自监督掩码建模方法，结合全局与局部对比损失，在十一个遥感基准上以单一通才模型超越多个专家模型。
摘要：
我们引入了一种高度多模态的 transformer，用于在空间和时间上表示多种遥感模态——多光谱光学、合成孔径雷达、高程、天气、伪标签等。这些输入对多种遥感任务很有用，例如作物制图和洪水检测。然而，学习遥感数据的共享表示具有挑战性，因为相关数据模态具有多样性，且感兴趣目标在尺度上差异巨大，从小型船只（1-2 像素且移动快速）到冰川（数千像素且变化缓慢）。我们提出了一种新颖的自监督学习算法，通过掩码建模在灵活的一组输入模态上提取多尺度特征。我们的双重全局与局部对比损失在目标（深层表示与浅层输入投影）和掩码策略（结构化与非结构化）上有所不同。我们的 Galileo 是一个单一的通才模型，在十一个基准和多个任务上优于卫星图像和像素时间序列的最先进专家模型。
英文摘要：
We introduce a highly multimodal transformer to represent many remote sensing modalities - multispectral optical, synthetic aperture radar, elevation, weather, pseudo-labels, and more - across space and time. These inputs are useful for diverse remote sensing tasks, such as crop mapping and flood detection. However, learning shared representations of remote sensing data is challenging, given the diversity of relevant data modalities, and because objects of interest vary massively in scale, from small boats (1-2 pixels and fast) to glaciers (thousands of pixels and slow). We present a novel self-supervised learning algorithm that extracts multi-scale features across a flexible set of input modalities through masked modeling. Our dual global and local contrastive losses differ in their targets (deep representations vs. shallow input projections) and masking strategies (structured vs. not). Our Galileo is a single generalist model that outperforms SoTA specialist models for satellite images and pixel time series across eleven benchmarks and multiple tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
