静止的注意力保持静止：打破视觉惯性以缓解认知幻觉
Attention at Rest Stays at Rest: Breaking Visual Inertia to Mitigate Relation Hallucinations
AI总结：
提出惯性感知视觉激发（IVE）方法，通过建模视觉注意力的动态响应性，打破多模态大模型中的视觉惯性，从而缓解需要对象间关系推理的认知幻觉。
摘要：
就像静止的物体保持静止一样，我们发现多模态大语言模型（MLLMs）中的视觉注意力表现出明显的惯性：一旦在早期解码步骤中稳定下来，注意力在很大程度上保持静态，无法支持认知推理所需的组合理解。虽然现有的幻觉缓解方法主要针对关于对象存在或属性的感知幻觉，但它们对于需要对象间关系推理的认知幻觉仍然不足。通过逐词元分析，我们将视觉惯性识别为一个促成因素：对语义关键区域的注意力持续集中，无法动态支持关系推理。因此，我们提出了惯性感知视觉激发（IVE），通过将认知推理建模为视觉注意力的动态响应性来打破这种惯性模式。具体来说，IVE选择相对于历史注意力趋势动态出现的视觉词元，同时区分表现出惯性行为的词元。为了进一步促进组合推理，IVE引入了惯性感知惩罚，阻止过度集中并限制注意力在局部区域内的持久性。大量实验表明，IVE在多种MLLMs和基准测试中无需额外训练即可有效工作。
英文摘要：
While multimodal large language models demonstrate strong entity-level perception, faithfully grounding relational interactions between objects remains a persistent challenge. Although conventional visual grounding techniques attempt to resolve hallucinations by amplifying visual attention, strengthening overall visual signals fails to reliably correct relational errors. Tracing visual attention in relation descriptions reveals that correct responses tend to dynamically shift focus across regions, whereas hallucinated responses often linger on previously dominant evidence, exhibiting an undesirable \textit{visual inertia}. Further analysis shows that relation-prediction performance steadily deteriorates as more previous-step visual attention is carried into the current decoding step. We therefore introduce Inertia-aware Visual Excitation (IVE), an MLLM decoding method that dynamically recalibrates visual values using token-level attention history. By contrasting current attention against recent moving averages, IVE separates emergent tokens with rising relevance from persistently dominant inertia tokens, selectively reinforcing newly needed evidence while mildly attenuating contributions from repeatedly attended regions. Across three MLLMs and decoding strategies, IVE reduces relation hallucinations while preserving broader multimodal performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
