纠正注意力分散引起的视觉模糊以减少幻觉：算法与理论
Correcting Visual Blur Induced by Attention Distraction to Reduce Hallucinations: Algorithm and Theory
AI总结：
本文揭示多模态大语言模型中的物体幻觉与类人注意力分散现象相关，并提出一种无需额外训练的注意力聚焦方法（AFIP）通过跨头注意力增强和动态历史注意力强化来纠正视觉模糊，从而减少幻觉。
摘要：
多模态大语言模型（MLLMs）经常遭受物体幻觉的困扰，但导致这一失败的视觉感知机制仍知之甚少。在这项工作中，我们揭示幻觉与一种类人注意力分散现象密切相关，其中人类在注意力分散下会经历视觉清晰度下降并产生不准确的描述，而在模型中，同样的机制表现为解码过程中多头注意力的空间不一致性以及对图像令牌注意力的时间衰减。我们进一步提供了理论见解，表明注意力分散会增加模型复杂度并降低分类泛化能力。受这些发现的启发，我们提出了一种用于改进图像感知的注意力聚焦方法（AFIP），该方法通过跨头注意力丰富来纠正注意力分散，并通过动态历史注意力增强来强化视觉基础。在多个基准和模型上的大量实验验证了AFIP的有效性，且无需额外训练。
英文摘要：
Multimodal large language models (MLLMs) frequently suffer from object hallucinations, yet the visual perceptual mechanism underlying this failure remains poorly understood. In this work, we reveal that hallucinations are strongly associated with a human-like attention distraction phenomenon, where humans under divided focus experience degraded visual clarity and produce inaccurate descriptions, while in models the same mechanism manifests as spatial inconsistency in multi-head attention and temporal fading of attention to image tokens during decoding. We further provide theoretical insights that attention dispersion increases model complexity and degrades classification generalization. Motivated by these findings, we propose an Attention-Focused Approach for Improved Image Perception (AFIP), which corrects attention distraction via cross-head attention enrichment and reinforces visual grounding through dynamic historical attention enhancement. Extensive experiments on multiple benchmarks and models validate the effectiveness of AFIP without additional training. Code is available at: https://github.com/MIKUZ12/AFIP.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
