重新思考对比解码：关于多模态大语言模型中对比解码在减轻对象幻觉方面无效性的可重复性研究与扩展
Rethinking the Effectiveness of Contrastive Decoding in Mitigating Hallucinations in MLLMs
浏览论文内容
中文总结 AI 辅助
研究多模态大语言模型中对比解码减轻对象幻觉的有效性，通过重现和扩展相关研究，进行多实验验证其在不同数据集上效果，发现其带来的改善常是虚假的，挑战了当前策略有效性，推动更可靠方法开发。
中文摘要 AI 辅助
对比解码（CD）已被提出作为一种无训练策略，用于减轻多模态大语言模型（MLLMs）中的对象幻觉，在诸如POPE基准测试中有收益报告。然而，近期工作质疑这些收益是否反映了视觉基础的真正改善。本研究重现并扩展了相关发现，测试CD在判别数据集中是否会引起单向输出分布偏移并检验其跨数据集的通用性，验证自适应合理性约束（APC）在判别和生成基准测试中能将采样减少到贪婪搜索。除了重现，还严格研究了CD在生成和判别数据集上的效果，通过多个实验提供了更多见解，实验结果验证了原观点，表明CD带来的明显改善往往是虚假的，不能持续转化为更强的视觉基础以减少幻觉。这些发现挑战了当前对比解码策略的有效性，并推动开发更可靠的方法来减轻MLLMs中的幻觉。
英文摘要
Multimodal large language models frequently describe objects that are not present in an image. Contrastive decoding has become a widely used inference-time remedy, and it is assumed to work by contrasting a normal forward pass against a hallucination-prone one so that hallucinated content is suppressed. However, whether the reported benchmark improvements actually arise from this mechanism has not been examined. Our key observation is that a correction which suppresses hallucination must depend on whether the model is hallucinating, and that this dependence can be measured directly inside the model. Following this idea, we analyse the internal computation of three contrastive decoding methods on both discriminative and generative tasks, and compare each against controls that remove the contrastive term while preserving its effect on the output. The correction turns out to be unrelated to hallucination at every layer, the gains in captioning come instead from a constraint that narrows sampling towards greedy decoding, and an unstructured perturbation of the same magnitude performs at least as well. Experiments across three models and standard hallucination benchmarks show that these improvements reflect a change in decoding behaviour rather than genuine hallucination mitigation.
发表机构
- Indian Institute of Technology, Roorkee（印度理工学院鲁尔基分校）
机构由 AI 辅助整理，请以论文原文为准。
