发表机构
University of Science and Technology of China; Hefei University of Technology; Lenovo Group, China; Chongqing University（中国科学技术大学; 合肥工业大学; 联想集团; 重庆大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多模态大模型情感幻觉，提出EHR评估器量化六个认知层面，并设计无需训练的HMER框架，通过记忆引导实现细粒度缓解，在19个模型上验证有效性。
AI 中文摘要
多模态大语言模型（MLLMs）在开放式情感理解方面展现出强大潜力，然而它们经常产生情感幻觉。评估此类幻觉尤其具有挑战性，原因有二。首先，情感理解涵盖多个认知层面，从多模态感知到心理推理。其次，情感解释以自由形式的语言表达，使得现有的封闭式评估协议不足以进行有效评估。为解决这些挑战，我们引入了EHR（情感幻觉率），一种在六个层面量化情感幻觉的评估器：表情、动作、音频、本能、逻辑和结论。利用EHR，我们发现现有的缓解方法往往在减少某些层面的幻觉的同时，却加剧了其他层面的幻觉，这暴露了粗粒度纠正的局限性以及对于层面感知定位和缓解的需求。受此发现启发，我们提出了HMER（幻觉感知的记忆引导情感推理），一个无需训练的情感幻觉缓解框架。HMER维护一个幻觉记忆，记录局部化的幻觉声明，并实现针对性的逻辑修正，同时还有一个锚定记忆，保留可靠的中间推理状态以稳定后续生成。通过选择性地抑制不可靠线索同时保留可信的推理上下文，HMER能够在不同幻觉层面实现细粒度的缓解。在19个MLLMs上的广泛实验证明了情感幻觉的普遍性以及我们框架在不同模型架构中的有效性。
英文摘要
Multimodal large language models (MLLMs) have shown strong potential in open-ended emotion understanding, yet they often generate emotion hallucinations. Evaluating such hallucinations is particularly challenging for two reasons. First, emotion understanding spans multiple cognitive facets, from multimodal perception to psychological reasoning. Second, emotional interpretations are expressed in free-form language, making existing closed-ended protocols insufficient for evaluation. To address these challenges, we introduce EHR (Emotion Hallucination Rate), an evaluator that quantifies emotion hallucinations across six facets: expression, action, audio, instinct, logic, and conclusion. Using EHR, we reveal that existing mitigation methods often reduce hallucinations in some facets while aggravating them in others, exposing the limitation of coarse-grained correction and the need for facet-aware localization and mitigation. Motivated by this finding, we propose HMER (Hallucination-aware Memory-guided Emotion Reasoning), a training-free framework for emotion hallucination mitigation. HMER maintains a Hallucination Memory that records localized hallucinated claims and enables targeted logit rectification, together with an Anchor Memory that preserves reliable intermediate reasoning states to stabilize subsequent generation. By selectively suppressing unreliable cues while preserving trustworthy reasoning context, HMER enables fine-grained mitigation across diverse hallucination facets. Extensive experiments on 19 MLLMs demonstrate the prevalence of emotion hallucinations and the effectiveness of our framework across diverse model architectures.
Comments10 pages, 6 figures
