大型视觉语言模型中间层的魔鬼：通过注意力透镜解释、检测和缓解对象幻觉
Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens
AI总结：
本文通过注意力透镜发现LVLMs中间层是处理视觉信息的关键阶段，并基于真实token与幻觉token的注意力差异提出一种推理时调整方法，无需额外训练即可有效缓解主流模型的对象幻觉。
摘要：
大型视觉语言模型（LVLMs）中的幻觉严重削弱了其可靠性，促使研究人员探索幻觉的成因。然而，大多数研究主要聚焦于语言方面而非视觉方面。本文研究LVLMs如何处理视觉信息，以及该过程是否引发幻觉。首先，我们使用注意力透镜识别LVLMs处理视觉数据的阶段，发现中间层至关重要。此外，我们发现这些层可进一步分为两个阶段：“视觉信息丰富”和“语义细化”，分别将视觉数据传播至对象token并通过文本进行解读。通过分析视觉信息丰富阶段的注意力模式，我们发现真实token始终比幻觉token获得更高的注意力权重，这是幻觉的强烈指示器。进一步检查多头注意力图显示，幻觉token通常源于与不一致对象交互的注意力头。基于这些见解，我们提出一种简单的推理时方法，通过整合各头信息来调整视觉注意力。大量实验表明，该方法无需额外训练成本即可有效缓解主流LVLMs中的幻觉。代码可在https://github.com/ZhangqiJiang07/middle_layers_indicating_hallucinations获取。
英文摘要：
Hallucinations in Large Vision-Language Models (LVLMs) significantly undermine their reliability, motivating researchers to explore the causes of hallucination. However, most studies primarily focus on the language aspect rather than the visual. In this paper, we address how LVLMs process visual information and whether this process causes hallucination. Firstly, we use the attention lens to identify the stages at which LVLMs handle visual data, discovering that the middle layers are crucial. Moreover, we find that these layers can be further divided into two stages: ''visual information enrichment'' and ''semantic refinement'' which respectively propagate visual data to object tokens and interpret it through text. By analyzing attention patterns during the visual information enrichment stage, we find that real tokens consistently receive higher attention weights than hallucinated ones, serving as a strong indicator of hallucination. Further examination of multi-head attention maps reveals that hallucination tokens often result from heads interacting with inconsistent objects. Based on these insights, we propose a simple inference-time method that adjusts visual attention by integrating information across various heads. Extensive experiments demonstrate that this approach effectively mitigates hallucinations in mainstream LVLMs without additional training costs. Code is available at https://github.com/ZhangqiJiang07/middle_layers_indicating_hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
