EmoMM：针对冲突和缺失性下的多模态情绪识别的MLLM基准测试与引导
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
AI总结：
本文提出EmoMM基准，用于研究多模态情绪识别中模态冲突和缺失性下的MLLM决策机制，发现视频贡献崩溃现象，并提出CHASE机制减轻决策偏差，提升模型在复杂情感场景中的可靠性。
摘要：
多模态情绪识别（MER）对于解读现实世界互动至关重要。尽管多模态大语言模型（MLLM）在MER中表现出色，但其在模态冲突和缺失性下的内部决策机制仍鲜有研究。本文为系统研究这些行为，引入EmoMM基准，包含对齐模态、冲突和缺失子集。通过广泛评估，发现视频贡献崩溃（VCC）现象，即MLLM因高token冗余和模态偏好而忽视视频证据。为解决此问题，提出冲突感知的头部级注意力引导（CHASE），一种轻量机制，可检测模态冲突并执行推理时的注意力引导，有效缓解决策偏差而不需重新训练主干。实验结果表明，CHASE在各种设置中均能提升性能，显著增强MLLM在复杂情感场景中的可靠性。
英文摘要：
Multimodal Emotion Recognition (MER) is critical for interpreting real-world interactions. While Multimodal Large Language Models (MLLM) have shown promise in MER, their internal decision-making mechanisms under modality conflict and missingness remain largely underexplored. In this paper, to systematically investigate these behaviors, we introduce EmoMM, a comprehensive benchmark featuring modality-aligned, conflict, and missing subsets. Through extensive evaluation, we uncover a Video Contribution Collapse (VCC) phenomenon, where MLLM marginalize video evidence due to high token redundancy and modality preferences. To address this, we propose Conflict-aware Head-level Attention Steering (CHASE), a lightweight mechanism that detects modality conflicts and performs inference-time attention steering, effectively mitigating decision bias without retraining the backbone. Experimental results demonstrate that CHASE consistently improves performance across various settings, significantly enhancing the reliability of MLLM in complex affective scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
