AI 中文总结
针对多模态学习中的模态不平衡问题，提出功能空间引导的多模态优化（FGMO），通过功能进展估计和功能响应控制实现跨模态协调优化，并在多个基准上验证有效性。
AI 中文摘要
多模态学习常常遭受模态不平衡问题，其中联合优化过程被单一模态主导。现有方法通常根据预测不确定性或优化统计得出的分数差异来估计模态不平衡。然而，由于不同模态在预测不确定性和学习动态上存在差异，直接比较此类分数可能将内在的模态差异误判为进展差距，导致不平衡估计出现偏差。本文提出功能空间引导的多模态优化（FGMO），利用功能空间进展信号评估各模态的优化进展，并协调跨模态优化以缓解模态不平衡。具体而言，我们引入功能进展估计（FPE）来度量每个模态更新引起的功能空间响应，并针对损失对齐的单模态参考进行校准，生成可比较的进展信号。基于该信号，功能响应控制（FRC）重新分配模态级功能空间预算，并通过张量级学习率调整实现目标响应。理论分析确立了在有界控制器状态失配下FRC的一步目标收缩性质，大量实验证明了FGMO在多个多模态基准上的有效性。
英文摘要
Multimodal learning often suffers from modality imbalance, where the joint optimization process is dominated by a single modality. Existing methods typically estimate modality imbalance from score disparities derived from prediction uncertainty or optimization statistics. However, due to distinct prediction uncertainty and learning dynamics across modalities, direct comparison of such scores may misinterpret intrinsic modality differences as progress gaps, leading to biased imbalance estimation. In this paper, we propose Function-Space Guided Multimodal Optimization (FGMO), which leverages a function-space progress signal to assess modality-wise optimization progress and coordinate optimization across modalities to alleviate modality imbalance. Specifically, we introduce Functional Progress Estimation (FPE) to measure each modality's update-induced function-space response and calibrate it against a loss-aligned unimodal reference, producing a comparable progress signal. Based on this signal, Functional Response Control (FRC) redistributes modality-level function-space budgets and realizes the target responses through tensor-wise learning-rate adjustment. Theoretical analysis establishes a one-step target-contraction property of FRC under bounded controller-state mismatch, and extensive experiments demonstrate the effectiveness of FGMO across multiple multimodal benchmarks.
