因果解码用于抗幻觉的多模态大语言模型
Causal Decoding for Hallucination-Resistant Multimodal Large Language Models
AI总结：
本文提出因果解码框架，通过针对性干预减少多模态大语言模型中的物体幻觉，提升响应的准确性和忠实度。
摘要：
多模态大语言模型（MLLMs）在视觉-语言任务中能够提供详细的响应，但仍然容易出现物体幻觉（在图像中不存在的物体被引入），这在实践中削弱了可靠性。先前的努力通常依赖于启发式惩罚、事后修正或通用解码调整，这些方法并未直接干预触发物体幻觉的机制，因此效果有限。为了解决这一挑战，我们提出了一种因果解码框架，该框架在生成过程中应用针对性的因果干预，以抑制虚假的物体提及。通过重塑解码动态以削弱虚假依赖，我们的方法在减少虚假物体标记的同时保持描述质量。在描述和问答基准测试中，我们的框架显著降低了物体幻觉率，并在不降低整体输出质量的情况下实现了最先进的忠实度。
英文摘要：
Multimodal Large Language Models (MLLMs) deliver detailed responses on vision-language tasks, yet remain susceptible to object hallucination (introducing objects not present in the image), undermining reliability in practice. Prior efforts often rely on heuristic penalties, post-hoc correction, or generic decoding tweaks, which do not directly intervene in the mechanisms that trigger object hallucination and thus yield limited gains. To address this challenge, we propose a causal decoding framework that applies targeted causal interventions during generation to curb spurious object mentions. By reshaping the decoding dynamics to attenuate spurious dependencies, our approach reduces false object tokens while maintaining descriptive quality. Across captioning and QA benchmarks, our framework substantially lowers object-hallucination rates and achieves state-of-the-art faithfulness without degrading overall output quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
