AI 中文总结
SynCo通过监督交互残差直接训练多模态协同，以可忽略的计算成本提升协同捕获，在Trifeature及真实基准上优于或匹配现有方法。
AI 中文摘要
多模态对比学习是从未标记数据中学习可迁移表示的主导范式，但标准目标主要捕获模态之间冗余的信息。部分信息分解（PID）表明，多模态数据中的任务相关信息可分解为三个组成部分：模态间共享的冗余、各模态特有的独特性，以及仅通过联合观察才能获得的协同。最近的框架扩展了对比学习以捕获所有三个组成部分，但协同在实践中仍训练不足。我们提出SynCo（协同对比学习），一种通过专门监督交互残差直接解决协同训练不足问题的方法。SynCo拟合一个线性投影器，从独立计算的单模态特征预测融合表示，由此产生的交互残差（去除了线性单模态可预测部分）以可忽略的计算成本接收专门的对比监督。在受控的Trifeature基准上，SynCo实现了最先进的协同捕获，相比基线有+5.98%的提升；在MultiBench、DARai和MM-IMDb的真实世界基准上，SynCo在多种模态组合和任务类型中持续优于或匹配先前方法。该方法作为现有对比多模态框架的插件运行，无需修改底层融合架构，并且与其他方法结合时可进一步提高协同捕获。
英文摘要
Multimodal contrastive learning is a dominant paradigm for learning transferable representations from unlabeled data, but standard objectives primarily capture information that is redundant between modalities. Partial Information Decomposition (PID) shows that task-relevant information in multimodal data decomposes into three components: redundancy shared between modalities, uniqueness specific to each modality, and synergy available only from their joint observation. Recent frameworks extend contrastive learning to capture all three components, yet synergy remains undertrained in practice. We propose SynCo (Synergy Contrastive Learning), a method that directly addresses synergy undertraining through dedicated supervision on an interaction residual. SynCo fits a linear projector to predict the fused representation from independently computed unimodal features, and the resulting interaction residual, which removes the linearly unimodal-predictable component, receives dedicated contrastive supervision at negligible computational cost. On the controlled Trifeature benchmark, SynCo achieves state-of-the-art synergy capture with a $+5.98\%$ gain over the baseline, and on real-world benchmarks from MultiBench, DARai, and MM-IMDb, SynCo consistently outperforms or matches prior methods across diverse modality combinations and task types. The method operates as a plug-in to existing contrastive multimodal frameworks without modifying the underlying fusion architecture and can further improve synergy capture when combined with other methods.
