arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2511.20258cs.CVcs.LG

模态平衡的协同蒸馏用于多模态领域泛化

Modality-Balanced Collaborative Distillation for Multi-Modal Domain Generalization

Xiaohan Wang, Zhangtao Cheng, Ting Zhong, Leiting Chen, Fan Zhou

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出MBCD框架，通过自适应模态丢弃、梯度一致性约束和跨模态蒸馏，解决多模态领域泛化中模态不平衡问题，提升模型泛化能力。

摘要：

加权平均（WA）已作为一种增强泛化能力的强大技术，通过促进收敛到平坦的损失景观，这与更强的分布外性能相关。然而，直接将WA应用于多模态领域泛化（MMDG）具有挑战性：不同模态的优化速度差异导致WA在早期阶段过度拟合于收敛更快的模态，抑制了较慢但互补的模态的贡献，从而阻碍了有效的模态融合，并使损失景观偏向更尖锐、泛化能力较差的极小值。为了解决这个问题，我们提出了MBCD，一种统一的协同蒸馏框架，保留了WA的平坦性诱导优势，同时克服了多模态环境中的不足。MBCD首先在学生模型中采用自适应模态丢弃，以抑制早期阶段对主导模态的偏见。随后，梯度一致性约束则在单模态分支和融合表示之间对齐学习信号，鼓励协调和更平滑的优化。最后，基于WA的教师通过将融合知识转移到每个单模态分支进行跨模态蒸馏，加强了跨模态交互，并引导收敛到更平坦的解决方案。在MMDG基准上的大量实验表明，MBCD在多模态领域泛化任务中始终优于现有方法，实现了在不同未见领域中更高的准确性和鲁棒性。

英文摘要：

Weight Averaging (WA) has emerged as a powerful technique for enhancing generalization by promoting convergence to a flat loss landscape, which correlates with stronger out-of-distribution performance. However, applying WA directly to multi-modal domain generalization (MMDG) is challenging: differences in optimization speed across modalities lead WA to overfit to faster-converging ones in early stages, suppressing the contribution of slower yet complementary modalities, thereby hindering effective modality fusion and skewing the loss surface toward sharper, less generalizable minima. To address this issue, we propose MBCD, a unified collaborative distillation framework that retains WA's flatness-inducing advantages while overcoming its shortcomings in multi-modal contexts. MBCD begins with adaptive modality dropout in the student model to curb early-stage bias toward dominant modalities. A gradient consistency constraint then aligns learning signals between uni-modal branches and the fused representation, encouraging coordinated and smoother optimization. Finally, a WA-based teacher conducts cross-modal distillation by transferring fused knowledge to each uni-modal branch, which strengthens cross-modal interactions and steer convergence toward flatter solutions. Extensive experiments on MMDG benchmarks show that MBCD consistently outperforms existing methods, achieving superior accuracy and robustness across diverse unseen domains.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑