IBD：通过图像偏置解码缓解大型视觉-语言模型中的幻觉
IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding
AI总结：
针对LVLMs因过度依赖语言先验而产生幻觉的问题，提出图像偏置解码IBD，通过对比常规与图像偏置模型的预测并自适应调整，在无需额外训练、参数增量极小的情况下显著提升回答真实性。
摘要：
尽管大型视觉-语言模型（LVLMs）取得了快速发展并得到广泛应用，但它们仍面临易于产生幻觉这一严峻挑战。过度依赖语言先验已被确定为导致这些幻觉的关键因素。本文提出通过引入一种新颖的图像偏置解码（IBD）技术来缓解该问题。该方法通过对比常规LVLM与图像偏置LVLM的预测来获得下一token概率分布，从而放大与图像内容高度相关的正确信息，同时减轻因过度依赖文本而产生的幻觉错误。我们进一步开展了全面的统计分析以验证方法的可靠性，并设计了自适应调整策略，以在不同条件下实现稳健且灵活的处理。跨多个评估指标的实验结果证实，尽管无需额外训练数据且模型参数仅有极少增加，该方法仍能显著减少LVLMs中的幻觉，并提升所生成回答的真实性。
英文摘要：
Despite achieving rapid developments and with widespread applications, Large Vision-Language Models (LVLMs) confront a serious challenge of being prone to generating hallucinations. An over-reliance on linguistic priors has been identified as a key factor leading to these hallucinations. In this paper, we propose to alleviate this problem by introducing a novel image-biased decoding (IBD) technique. Our method derives the next-token probability distribution by contrasting predictions from a conventional LVLM with those of an image-biased LVLM, thereby amplifying the correct information highly correlated with image content while mitigating the hallucinatory errors caused by excessive dependence on text. We further conduct a comprehensive statistical analysis to validate the reliability of our method, and design an adaptive adjustment strategy to achieve robust and flexible handling under varying conditions. Experimental results across multiple evaluation metrics verify that our method, despite not requiring additional training data and only with a minimal increase in model parameters, can significantly reduce hallucinations in LVLMs and enhance the truthfulness of the generated response.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
