Fus-MAE：一种用于遥感中掩码自编码器的基于交叉注意力的数据融合方法
Fus-MAE: A cross-attention-based data fusion approach for Masked Autoencoders in remote sensing
AI总结：
针对遥感多模态数据融合中对比学习依赖复杂数据增强的问题，提出基于交叉注意力的掩码自编码器框架Fus-MAE，实现SAR与光学数据的有效融合，性能比肩定制化对比学习方法且优于其他同类MAE框架。
摘要：
用于表征学习的自监督框架近期引发了遥感领域的关注，因为它们有望降低整理大型卫星图像数据集所需的高昂标注成本。在多模态数据融合领域，尽管常用的对比学习方法有助于弥合不同传感器类型之间的域差距，但它们依赖的数据增强技术需要专业知识和精心设计，尤其是对于多光谱遥感数据而言。规避这些局限的一种可行但研究甚少的途径是采用基于掩码图像建模的预训练策略。本文提出了Fus-MAE，这是一种基于掩码自编码器的自监督学习框架，它利用交叉注意力在合成孔径雷达（SAR）与多光谱光学数据这两种存在显著域差距的模态之间执行早期及特征级数据融合。我们的实验结果表明，Fus-MAE的性能可与专为SAR-光学数据融合定制的对比学习策略相媲美，且优于在更大语料库上训练的其他掩码自编码器框架。
英文摘要：
Self-supervised frameworks for representation learning have recently stirred up interest among the remote sensing community, given their potential to mitigate the high labeling costs associated with curating large satellite image datasets. In the realm of multimodal data fusion, while the often used contrastive learning methods can help bridging the domain gap between different sensor types, they rely on data augmentations techniques that require expertise and careful design, especially for multispectral remote sensing data. A possible but rather scarcely studied way to circumvent these limitations is to use a masked image modelling based pretraining strategy. In this paper, we introduce Fus-MAE, a self-supervised learning framework based on masked autoencoders that uses cross-attention to perform early and feature-level data fusion between synthetic aperture radar and multispectral optical data - two modalities with a significant domain gap. Our empirical findings demonstrate that Fus-MAE can effectively compete with contrastive learning strategies tailored for SAR-optical data fusion and outperforms other masked-autoencoders frameworks trained on a larger corpus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
