AI 中文总结
该研究针对大型视觉语言模型的对象幻觉问题，提出基于内在 grounding 特征（IGS）的验证器引导解码（VGD）框架，在保留视觉理解和对象覆盖率的同时，实现了最先进的对象幻觉缓解效果。
AI 中文摘要
大型视觉语言模型（LVLMs）常对图像中不存在的对象产生幻觉。尽管已有进展，现有缓解方法仍缺乏可靠的对象级 grounding 诊断，因此倾向于应用粗粒度干预，这会损害视觉理解、缩短响应并降低真正 grounding 对象的覆盖率。关键挑战在于生成过程中检测每个新出现的对象提及是否有可靠视觉证据支持，以便选择性缓解幻觉。然而输出置信度反映的是下一个 token 的合理性而非视觉支持，使得语言先验让不存在的对象显得确定。我们表明，缺失的诊断证据编码在 Intrinsic Grounding Signature（IGS，一种分布式符号注意力模式，对这类自信幻觉仍具信息性）中。基于 IGS，我们提出 Verifier-Guided Decoding（VGD），这是一种解码框架：轻量验证器检查每个新出现的对象提及，当提及被识别为高风险时回滚 KV 缓存，抑制该对象及其同义词并重新生成受影响的后续内容。由于 VGD 仅对高风险对象提及进行干预，它在减少对象幻觉的同时保留模型原有的视觉理解和 grounding 对象覆盖率。在 CHAIR 和 AMBER-G 上的实验表明，VGD 实现了最先进的对象幻觉减少：在 @rec90 时，它将 AMBER-G 的 CHAIR 指标降低 43.6%，同时保留 99.6% 的 grounding 对象覆盖率；并将 CHAIR-MSCOCO 的 CHAIR_i/CHAIR_s 指标分别降低 37.0%/30.4%，且不缩短描述文本。
英文摘要
Large vision-language models (LVLMs) often hallucinate objects that are absent from an image. Despite recent progress, existing mitigation methods still lack reliable object-level grounding diagnostics and therefore tend to apply coarse-grained interventions, which can impair visual understanding, shorten responses, and reduce coverage of genuinely grounded objects. The key challenge is thus to detect, during generation, whether each emerging object mention is supported by reliable visual evidence, so that hallucination can be mitigated selectively. Yet output confidence reflects next-token plausibility rather than visual support, allowing language priors to make absent objects appear certain. We show that the missing diagnostic evidence is encoded in an Intrinsic Grounding Signature (IGS), a distributed signed attention pattern that remains informative for such confident hallucinations. Based on IGS, we propose Verifier-Guided Decoding (VGD), a decoding framework in which a lightweight verifier examines each emerging object mention, rolls back the KV cache when the mention is identified as high risk, suppresses the object and its synonyms, and regenerates the affected continuation. Because VGD intervenes only on object mentions identified as high risk, it reduces object hallucination while preserving the model's original visual understanding and grounded object coverage. Experiments on CHAIR and AMBER-G show that VGD achieves state-of-the-art object hallucination reduction: at @rec90, it cuts AMBER-G CHAIR by 43.6\% while retaining 99.6\% of grounded-object coverage, and reduces CHAIR-MSCOCO CHAIR$_i$/CHAIR$_s$ by 37.0\%/30.4\% without shortening captions.
