更深入的思考，更弱的目标：理解并缓解多模态大语言模型推理过程中感知障碍
Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language Models
AI总结：
本文研究了多模态大语言模型在推理过程中出现的视觉感知障碍问题，提出了一种无需训练的视觉区域引导注意力框架，通过选择和重新加权视觉头部来引导模型关注与问题相关区域，从而提高视觉定位和推理准确性。
摘要：
多模态大语言模型（MLLMs）在进行扩展推理模式时常常出现感知障碍，特别是在视觉问答（VQA）任务中。我们识别出注意力分散是根本原因：在多步推理过程中，模型的视觉注意力变得分散并远离与问题相关区域，实际上“失去焦点”于视觉输入。为了更好地理解这一现象，我们分析了MLLMs的注意力图，并观察到推理提示显著减少了回答问题关键区域的注意力。我们进一步发现模型对图像标记的总体注意力与图像内注意力的空间分散性之间存在强相关性。基于这一见解，我们提出了一个无需训练的视觉区域引导注意力（VRGA）框架，该框架根据熵-聚焦准则选择视觉头部并重新加权其注意力，从而有效引导模型在推理过程中关注与问题相关区域。在视觉-语言基准上的广泛实验表明，我们的方法有效缓解了感知退化，从而在视觉定位和推理准确性方面取得改进，同时提供了可解释的见解，说明MLLMs如何处理视觉信息。
英文摘要：
Multimodal large language models (MLLMs) often suffer from perceptual impairments under extended reasoning modes, particularly in visual question answering (VQA) tasks. We identify attention dispersion as the underlying cause: during multi-step reasoning, the model's visual attention becomes scattered and drifts away from question-relevant regions, effectively "losing focus" on the visual input. To better understand this phenomenon, we analyze the attention maps of MLLMs and observe that reasoning prompts significantly reduce attention to regions critical for answering the question. We further find a strong correlation between the model's overall attention on image tokens and the spatial dispersiveness of its attention within the image. Leveraging this insight, we propose a training-free Visual Region-Guided Attention (VRGA) framework that selects visual heads based on an entropy-focus criterion and reweights their attention, effectively guiding the model to focus on question-relevant regions during reasoning. Extensive experiments on vision-language benchmarks demonstrate that our method effectively alleviates perceptual degradation, leading to improvements in visual grounding and reasoning accuracy while providing interpretable insights into how MLLMs process visual information.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
