VGS-Decoding：基于视觉 grounding 分数的解码方法用于医疗 VLM 中的幻觉抑制
VGS-Decoding: Visual Grounding Score Guided Decoding for Hallucination Mitigation in Medical VLMs
AI总结：
本文提出 VGS-Decoding 方法，通过视觉 grounding 分数指导解码以减少医疗 VLM 的幻觉问题，实验表明在多个数据集上取得了显著提升，且仅引入2倍推理开销。
摘要：
医疗视觉-语言模型（VLMs）常因基于语言先验生成响应而非视觉证据而产生幻觉，这在临床应用中存在风险。我们提出了视觉 grounding 分数指导解码（VGS-Decoding），一种无需训练的推理缓解方法。我们的关键见解是，幻觉标记在视觉信息退化时维持或增加其概率，而视觉 grounding 标记则降低。我们引入了视觉 grounding 分数（VGS），通过比较原始和退化图像的分布来衡量每个标记的视觉依赖性。在解码过程中，我们通过放大视觉 grounding 标记并抑制幻觉来重新加权概率。与固定权重对比方法不同，VGS-Decoding 提供了每标记自适应控制。在 MIMIC-Diff-VQA 和 VQA-RAD 上的实验表明，相对于 LLaVA-Med、CheXagent 和 MedGemma，VGS-Decoding 在多个数据集上均取得一致提升，整体提升达 +9.12%，开放性回忆提升达 +8.98%，且仅引入 2 倍推理开销，无需额外训练，适合临床部署。接受后将公开代码以促进可重复性。
英文摘要：
Medical Vision-Language Models (VLMs) often hallucinate by generating responses based on language priors rather than visual evidence, posing risks in clinical applications. We propose Visual Grounding Score Guided Decoding (VGS-Decoding), a training-free method to mitigate hallucinations during inference. Our key insight is that hallucinated tokens maintain or increase their probability when visual information is degraded, while visually grounded tokens decrease in probability. We introduce the Visual Grounding Score (VGS), which measures each token's visual dependency by comparing distributions from original and distorted images. During decoding, we reweight probabilities by amplifying visually grounded tokens while suppressing hallucinations. Unlike fixed-weight contrastive methods, VGS-Decoding provides per-token adaptive control. Experiments on MIMIC-Diff-VQA and VQA-RAD across LLaVA-Med, CheXagent, and MedGemma demonstrate consistent improvements, with up to +9.12% overall gain and $+8.98\%$ in open-ended recall, while introducing only $2\times$ inference overhead and no additional training, making it practical for clinical deployment. Upon acceptance, code will be released publicly to facilitate reproducibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
