消除语义漂移：一种动态方法用于在大视觉-语言模型中进行生成基础
Curing Semantic Drift: A Dynamic Approach to Grounding Generation in Large Vision-Language Models
AI总结：
本文提出DLC方法，通过动态校准logits来减少大视觉-语言模型中的语义漂移，提升生成质量。
摘要：
大型视觉-语言模型（LVLMs）面临语言先验与视觉证据之间的拉锯，常导致语义漂移：一种逐步脱离输入图像的现象。通过逐个token诊断，发现幻觉并非由缺乏基础候选词引起，而是由选择失败导致——模型在存在更基础的替代词时，仍选择语言上方便但视觉上不忠实的token。受此启发，我们提出动态logits校准（DLC），一种无需训练的解码框架，引入轻量级视觉 referee 在漂移发生时介入。在每一步，DLC对top-k候选词进行双方面基础检查：（1）评估候选词的内在视觉相关性；（2）其上下文视觉一致性。这些信号与自适应历史基线对比，计算相对视觉优势，进而动态校准logits，优先选择基础token。在CHAIR、POPE、SHR、GPT-4o评估和MME上的广泛实验表明，DLC在多个LVLMs中一致减少幻觉，同时保持响应质量。进一步分析验证了对不同视觉backbone的鲁棒性，并展示了候选池大小变化时输出质量与计算成本之间的有利权衡。代码将在https://github.com/JiaheChen2002/DLC上发布。
英文摘要：
Large Vision-Language Models (LVLMs) face a tug-of-war between powerful linguistic priors and visual evidence, often leading to \emph{semantic drift}: a progressive detachment from the input image that can abruptly emerge at specific decoding steps. Through a token-level diagnosis, we show that hallucination is frequently triggered not by the absence of grounded candidates, but by a failure of selection -- the model chooses a linguistically convenient yet visually unfaithful token even when better grounded alternatives exist. Motivated by this insight, we propose \textbf{D}ynamic \textbf{L}ogits \textbf{C}alibration (DLC), a training-free decoding framework that introduces a lightweight visual referee to intervene exactly when drift happens. At each step, DLC performs a dual-aspect grounding check on top-$k$ candidates: (1) it assesses the intrinsic visual relevance of a candidate token and (2) its contextual visual coherence. These signals are evaluated against an adaptive historical baseline to compute a relative visual advantage, which is then used to dynamically calibrate logits and favor grounded tokens. Extensive experiments on CHAIR, POPE, SHR, GPT-4o evaluation, and MME demonstrate that DLC consistently reduces hallucinations across multiple LVLMs while preserving response quality. Further analyses validate robustness to different vision backbones and demonstrate a favorable trade-off between output quality and computational cost as the candidate pool size varies. Code will be released on https://github.com/JiaheChen2002/DLC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
