平衡多模态情感分析的幻象：超越基于优化的方法之局限
The Illusion of Balanced Multimodal Sentiment Analysis: Beyond the Limits of Optimization-Based Methods
- Mines Paris-PSL University（巴黎高科矿业学院）
- University of Bern（伯尔尼大学）
- National Technical University of Athens（雅典国立技术大学）
- Archimedes AI（阿基米德人工智能公司）
- Synaptic Bloom PBC（Synaptic Bloom公益公司）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文揭示多模态情感分析中基于优化的平衡方法失效，提出统一评估框架、理论诊断及留出判别性模态价值评估议程，实验表明现有策略均不稳定优于晚期拼接。
AI中文摘要：
多模态情感分析（MSA）仍受模态不平衡的制约，然而该领域继续依赖基于优化的平衡方法，这些方法承诺的效果远超其实际表现。我们做出三项贡献：1）一个统一的评估框架，在受控设置下测试基于梯度和基于损失的平衡策略；2）一项理论诊断，解释这些方法为何失败，因为它们混淆了拟合速度与判别贡献；3）一项研究议程，旨在实现基于留出数据的判别性模态价值评估。在CMU-MOSI和CMU-MOSEI上的实验揭示了三个缺陷：没有策略能稳定优于晚期拼接；性能对超参数敏感；甚至比率校准也无法产生一致的收益。核心问题在于根本层面：损失并非效用，梯度并非重要性。模态不平衡仍未解决，这促使从留出性能中估计效用。
英文摘要：
Multimodal Sentiment Analysis (MSA) remains constrained by modality imbalance, yet the field continues to rely on optimization-based balancing methods that promise more than they deliver. We provide three contributions: 1) a unified evaluation framework testing gradient and loss-based balancing strategies under controlled settings; 2) a theoretical diagnosis explaining why these methods fail, as they conflate fitting speed with discriminative contribution; and 3) a research agenda toward held-out discriminative modality valuation. Experiments on CMU-MOSI and CMU-MOSEI reveal three shortcomings: no strategy reliably outperforms Late Concatenation; performance is sensitive to hyperparameters; and even ratio calibration fails to yield consistent gains. The core issue is fundamental: loss is not utility, and gradients are not importance. Modality imbalance remains unresolved, motivating utility estimation from held-out performance.
