通过链看穿：通过CoT压缩和对比偏好优化缓解多模态推理模型的幻觉
Seeing Through the Chain: Mitigate Hallucination in Multimodal Reasoning Models via CoT Compression and Contrastive Preference Optimization
AI总结：
本文提出C3PO框架，通过CoT压缩和对比偏好优化缓解多模态推理模型的幻觉问题，通过过滤冗余思考 token 提升信号效率，并利用高质量反馈和定制诱导器增强对比学习效果。
摘要：
尽管多模态推理模型（MLRMs）展现了 impressive 能力，但它们仍易产生幻觉，有效解决方案仍待探索。本文通过实验分析幻觉成因，提出C3PO框架，包含CoT压缩和对比偏好优化。首先，我们发现引入推理机制会加重模型对语言先验的依赖，而忽视视觉输入，导致生成的CoT包含较少视觉线索但冗余文本token。为此，我们提出选择性过滤冗余思考token，以获得更紧凑且信号高效的CoT表示，保留任务相关信息并抑制噪声。此外，我们观察到推理轨迹的质量很大程度上决定了后续响应中幻觉是否产生。为此，我们引入了增强推理的偏好微调方案，通过高质量AI反馈构建训练对。我们进一步设计了多模态幻觉诱导机制，通过精心设计的诱导器激发模型的内在幻觉模式，产生具有信息量的负样本信号用于对比纠正。我们提供了理论依据证明有效性，并在多样化的MLRMs和基准上展示了持续的幻觉减少。
英文摘要：
While multimodal reasoning models (MLRMs) have exhibited impressive capabilities, they remain prone to hallucinations, and effective solutions are still underexplored. In this paper, we experimentally analyze the hallucination cause and propose C3PO, a training-based mitigation framework comprising \textbf{C}hain-of-Thought \textbf{C}ompression and \textbf{C}ontrastive \textbf{P}reference \textbf{O}ptimization. Firstly, we identify that introducing reasoning mechanisms exacerbates models' reliance on language priors while overlooking visual inputs, which can produce CoTs with reduced visual cues but redundant text tokens. To this end, we propose to selectively filter redundant thinking tokens for a more compact and signal-efficient CoT representation that preserves task-relevant information while suppressing noise. In addition, we observe that the quality of the reasoning trace largely determines whether hallucination emerges in subsequent responses. To leverage this insight, we introduce a reasoning-enhanced preference tuning scheme that constructs training pairs using high-quality AI feedback. We further design a multimodal hallucination-inducing mechanism that elicits models' inherent hallucination patterns via carefully crafted inducers, yielding informative negative signals for contrastive correction. We provide theoretical justification for the effectiveness and demonstrate consistent hallucination reduction across diverse MLRMs and benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
