超越全局评分：细粒度令牌接地作为检测LVLM幻觉的稳健检测器
Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations
AI总结：
本文提出细粒度令牌接地方法，通过分析模型层间令牌交互，发现幻觉令牌的扩散注意力模式和语义对齐失败特征，实现90%的幻觉检测准确率。
摘要：
大型视觉-语言模型（LVLMs）在视觉推理任务中表现优异，但易受幻觉影响。现有检测方法依赖于整体图像层面的对象令牌相关性测量，但这种全局策略受限：幻觉令牌可能在许多局部区域表现出弱但广泛散播的相关性，导致整体相关性高而逃避当前检测。我们提出基于细粒度令牌交互的检测框架，发现幻觉令牌的两个特征：（i）产生扩散的非局部注意力模式，与忠实令牌的紧凑注意力不同；（ii）无法与任何视觉区域形成有意义的语义对齐。基于这些发现，我们开发了一种轻量且可解释的检测方法，结合补丁级统计特征和隐藏层表示，实现90%的令牌级幻觉检测准确率，证明细粒度结构分析在检测幻觉中的优越性。
英文摘要：
Large vision-language models (LVLMs) achieve strong performance on visual reasoning tasks but remain highly susceptible to hallucination. Existing detection methods predominantly rely on coarse, whole-image measures of how an object token relates to the input image. This global strategy is limited: hallucinated tokens may exhibit weak but widely scattered correlations across many local regions, which aggregate into deceptively high overall relevance, thus evading the current global hallucination detectors. We begin with a simple yet critical observation: a faithful object token must be strongly grounded in a specific image region. Building on this insight, we introduce a patch-level hallucination detection framework that examines fine-grained token-level interactions across model layers. Our analysis uncovers two characteristic signatures of hallucinated tokens: (i) they yield diffuse, non-localized attention patterns, in contrast to the compact, well-focused attention seen in faithful tokens; and (ii) they fail to exhibit meaningful semantic alignment with any visual region. Guided by these findings, we develop a lightweight and interpretable detection method that leverages patch-level statistical features, combined with hidden-layer representations. Our approach achieves up to 90% accuracy in token-level hallucination detection, demonstrating the superiority of fine-grained structural analysis for detecting hallucinations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
