学习优先关注什么：模态重要性引导的推理用于可靠的多模态情感理解
Learning What to Attend First: Modality-Importance-Guided Reasoning for Reliable Multimodal Emotion Understanding
AI总结：
本文提出MIGR框架，通过模态重要性引导推理，提升多模态情感理解的可靠性，实验表明能有效减少情感不一致的解释实例。
摘要：
在本文中，我们提出了模态重要性引导推理（MIGR），一种框架，旨在提高基于推理的多模态情感理解在多模态大语言模型中的可靠性。尽管现有方法在情感理解方面有所进步，但它们往往受到推理漂移的影响：模型逐渐依赖于自身生成的文本而不是多模态证据，且其解释过于受视觉引导的推理路径影响。为了解决这些问题，我们引入了模态重要性（MI），一种简单而有效的机制，用于识别情感主导的模态。利用MI，MIGR重新组织推理序列，使解释从对目标情感最关键的模态开始，防止早期推理被信息较少的线索误导。我们的两阶段框架，包括模态对齐的监督微调和模态感知的奖励优化，鼓励模型生成情感基础、因果相关且保持连贯性的解释。在DFEW基准测试中的实验结果表明，MIGR显著提高了推理可靠性，将正确预测但伴随情感不一致解释的实例从18.10%降低到7.37%。这些结果证实了从情感主导模态开始推理的好处。
英文摘要：
In this paper, we present Modality-Importance-Guided Reasoning (MIGR), a framework designed to improve the reliability of reasoning-based multimodal emotion understanding in multimodal large language models. Although existing methods have advanced emotion understanding, they often suffer from reasoning drift: models gradually rely on their own generated text instead of multimodal evidence, and their explanations are overly shaped by visually initiated reasoning paths. To address these issues, we introduce Modality Importance (MI), a simple yet effective mechanism for identifying the emotion-dominant modality. Using MI, MIGR reorganizes reasoning sequences so that explanations begin from the modality most critical to the target emotion, preventing early reasoning from being misled by less informative cues. Our two-stage framework-comprising modality-aligned supervised fine-tuning and modality-aware reward optimization-encourages models to generate emotionally grounded, causally relevant, and coherence-preserving explanations. Experimental results on the DFEW benchmark show that MIGR substantially improves reasoning reliability, decreasing instances of correct predictions accompanied by emotionally inconsistent explanations from 18.10% to 7.37%. These results confirm the benefit of initiating reasoning from the emotion-dominant modality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
