更少的协同，更好的表现：通过条件和降质掩码自编码器解决异质多模态图像联合预训练
Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencoder
AI总结：
本文提出CoDe-MAE，通过Optical-anchored Knowledge Distillation和Conditioned Contrastive Learning解决高分辨率多模态联合预训练中的异质性-分辨率悖论，有效防止表示退化并在多个下游任务中取得新突破。
摘要：
在极异质的模态间学习稳健的表示仍然是多模态视觉中的基本挑战。作为这一挑战的重要且深刻的实例，高分辨率（HR）联合光学和合成孔径雷达（SAR）预训练旨在通过模态协同来互相增强单源表示；其潜力受到异质性-分辨率悖论的严重阻碍：更精细的空间尺度会大幅增加复杂雷达几何与非同源光学纹理之间的物理差异。因此，将中分辨率导向的刚性对齐范式迁移到高分辨率场景会引发要么严重的特征抑制以强制等同，要么由极端知识不确定性驱动的特征污染。两者极端不可避免地导致深刻的表示退化和负迁移。为克服这一瓶颈，我们提出CoDe-MAE，开创一种'更好的协同，更少的对齐'哲学。首先，Optical-anchored Knowledge Distillation（OKD）通过将SAR的斑点噪声映射到纯语义流形来隐式正则化。在此基础上，Conditioned Contrastive Learning（CCL）利用梯度缓冲机制对齐共享共识，同时安全地保留分歧的物理特征。同时，Cross-Modal Degraded Reconstruction（CDR）有意剥离非同源的光谱伪特征，截断本质上不恰当的映射，以捕捉真正的结构不变量。广泛的分析验证了我们的理论主张。在100万样本上预训练的CoDe-MAE展示了显著的数据效率，成功防止了表示退化，并在多种单模和双模下游任务中建立了新的最先进的性能，大幅超越了在远大数据集上扩展的基础模型。
英文摘要：
Learning robust representations across extremely heterogeneous modalities remains a fundamental challenge in multi-modal vision. As a critical and profound instantiation of this challenge, high-resolution (HR) joint optical and synthetic aperture radar (SAR) pretraining seeks modality synergy to mutually enhance single-source representations; its potential is severely hindered by the Heterogeneity-Resolution Paradox: finer spatial scales drastically amplify the physical divergence between complex radar geometries and non-homologous optical textures. Consequently, migrating medium-resolution-oriented rigid alignment paradigms to HR scenarios triggers either severe feature suppression to force equivalence, or feature contamination driven by extreme epistemic uncertainty. Both extremes inevitably culminate in profound representation degradation and negative transfer. To overcome this bottleneck, we propose CoDe-MAE, pioneering a \textit{better synergy with less alignment} philosophy. First, Optical-anchored Knowledge Distillation (OKD) implicitly regularizes SAR's speckle noise by mapping it into a pure semantic manifold. Building on this, Conditioned Contrastive Learning (CCL) utilizes a gradient buffering mechanism to align shared consensus while safely preserving divergent physical signatures. Concurrently, Cross-Modal Degraded Reconstruction (CDR) deliberately strips non-homologous spectral pseudo-features, truncating the inherently ill-posed mapping to capture true structural invariants. Extensive analyses validate our theoretical claims. Pretrained on 1M samples, CoDe-MAE demonstrates remarkable data efficiency, successfully preventing representation degradation and establishing new state-of-the-art performance across diverse single- and bi-modal downstream tasks, substantially outperforming foundation models scaled on vastly larger datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
