arXiv 2410.045142025-11-05cs.CLcs.CV

DAMRO：深入LVLM的注意力机制以减少对象幻觉

DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination

Department of Computer Science and Technology, Tongji University（计算机科学与技术系，同济大学）

Xuan Gong, Tianshi Ming, Xinpeng Wang, Zhihua Wei

备注：: Accepted by EMNLP2024 (Main Conference), add GitHub link

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出免训练方法DAMRO，通过利用ViT的CLS token过滤背景高注意力离群token，在解码阶段消除其影响，从而有效缓解LVLMs的对象幻觉问题。

摘要：

尽管大型视觉语言模型（LVLMs）取得了巨大成功，但它们不可避免地遭受幻觉问题。众所周知，LVLMs中的视觉编码器和大型语言模型（LLM）解码器均基于Transformer，使模型能够通过注意力机制提取视觉信息并生成文本输出。我们发现，LLM解码器在图像token上的注意力分布与视觉编码器高度一致，且两者分布都倾向于关注图像中特定的背景token，而非被指称的对象。我们将这种意外的注意力分布归因于视觉编码器本身固有的缺陷，该缺陷误导LLMs过度强调冗余信息并产生对象幻觉。为了解决这一问题，我们提出DAMRO，一种新颖的免训练策略，即深入LVLM的注意力机制以减少对象幻觉（Dive into Attention Mechanism of LVLM to Reduce Object Hallucination）。具体而言，我们的方法利用ViT的分类token（CLS）过滤掉散布在背景中的高注意力离群token，然后在解码阶段消除它们的影响。我们在包括LLaVA-1.5、LLaVA-NeXT和InstructBLIP在内的LVLMs上评估了我们的方法，使用了POPE、CHAIR、MME和GPT-4V辅助评估等多种基准。结果表明，我们的方法显著降低了这些离群token的影响，从而有效缓解了LVLMs的幻觉问题。代码已发布于https://github.com/coder-gx/DAMRO。

英文摘要：

Despite the great success of Large Vision-Language Models (LVLMs), they inevitably suffer from hallucination. As we know, both the visual encoder and the Large Language Model (LLM) decoder in LVLMs are Transformer-based, allowing the model to extract visual information and generate text outputs via attention mechanisms. We find that the attention distribution of LLM decoder on image tokens is highly consistent with the visual encoder and both distributions tend to focus on particular background tokens rather than the referred objects in the image. We attribute to the unexpected attention distribution to an inherent flaw in the visual encoder itself, which misguides LLMs to over emphasize the redundant information and generate object hallucination. To address the issue, we propose DAMRO, a novel training-free strategy that $D$ive into $A$ttention $M$echanism of LVLM to $R$educe $O$bject Hallucination. Specifically, our approach employs classification token (CLS) of ViT to filter out high-attention outlier tokens scattered in the background and then eliminate their influence during decoding stage. We evaluate our method on LVLMs including LLaVA-1.5, LLaVA-NeXT and InstructBLIP, using various benchmarks such as POPE, CHAIR, MME and GPT-4V Aided Evaluation. The results demonstrate that our approach significantly reduces the impact of these outlier tokens, thus effectively alleviating the hallucination of LVLMs. The code is released at https://github.com/coder-gx/DAMRO.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。