VERA：识别和利用长上下文理解中的视觉证据检索头
VERA: Identifying and Leveraging Visual Evidence Retrieval Heads in Long-Context Understanding
AI总结：
VERA通过识别和利用视觉证据检索头，提升长上下文理解能力，显著提高开源VLMs的性能。
摘要：
尽管视觉-语言模型（VLMs）在文本理解方面表现出色，但在处理长上下文和复杂推理任务时面临重大挑战。在本文中，我们分析了VLMs在长上下文处理中的内部机制，以理解其性能瓶颈。通过注意力分析的视角，我们识别出特定的视觉证据检索（VER）头——一组稀疏且动态的注意力头，这些头在推理过程中用于定位视觉线索，不同于静态的OCR头。我们证明这些头是模型性能的原因；屏蔽它们会导致显著的退化。基于这一发现，我们提出了VERA（视觉证据检索增强），一个无需训练的框架，该框架通过检测模型不确定性（即熵）来触发由VER头关注的视觉证据的显式 verbalization。全面的实验表明，VERA显著提高了开源VLMs的长上下文理解能力：在五个基准测试中，它在Qwen3-VL-8B-Instruct上平均相对改进为21.3%，在GLM-4.1V-Thinking上为20.1%。
英文摘要：
While Vision-Language Models (VLMs) have shown promise in textual understanding, they face significant challenges when handling long context and complex reasoning tasks. In this paper, we dissect the internal mechanisms governing long-context processing in VLMs to understand their performance bottlenecks. Through the lens of attention analysis, we identify specific Visual Evidence Retrieval (VER) Heads - a sparse, dynamic set of attention heads critical for locating visual cues during reasoning, distinct from static OCR heads. We demonstrate that these heads are causal to model performance; masking them leads to significant degradation. Leveraging this discovery, we propose VERA (Visual Evidence Retrieval Augmentation), a training-free framework that detects model uncertainty (i.e., entropy) to trigger the explicit verbalization of visual evidence attended by VER heads. Comprehensive experiments demonstrate that VERA significantly improves long-context understanding of open-source VLMs: it yields an average relative improvement of 21.3% on Qwen3-VL-8B-Instruct and 20.1% on GLM-4.1V-Thinking across five benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
