arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2508.00378cs.AIcs.CV

CoRGI:带事后视觉锚定的可验证链式思维推理

CoRGI: Verified Chain-of-Thought Reasoning with Post-hoc Visual Grounding

Shixin Yi, Lin Shang

AI总结:

CoRGI通过事后将链式思维逐步锚定到视觉证据并过滤无支撑声明,减少VLM幻觉,在五个基准上提升准确性与解释忠实度。

摘要:

视觉-语言模型(VLM)的多模态推理常常遭受幻觉问题,因为模型往往在仅对图像进行表面检查后就生成解释。我们提出CoRGI(带锚定洞察的链式推理),一个通过事后验证链式思维输出来增强推理可靠性的框架。给定VLM生成的推理依据,CoRGI将其分解为逐步陈述,将每一步锚定到视觉证据中,并在生成最终答案前过滤或纠正无支撑的声明。在五个具有挑战性的基准——VCR、ScienceQA、MMMU、MathVista和HallusionBench上的实验表明,CoRGI在多个VLM骨干网络(包括Qwen-2.5VL、LLaVA-1.6和Gemma3-12B)上持续提升了答案准确性和解释忠实度。除了定量提升外,定性分析进一步展示了验证过程如何减少幻觉并增强可解释性,表明事后视觉锚定是构建更可信、更透明的多模态推理系统的一个有前景的方向。

英文摘要:

Multimodal reasoning with vision-language models (VLMs) often suffers from hallucinations, as models tend to generate explanations after only a superficial inspection of the image. We present \textbf{CoRGI}(\textbf{C}hain \textbf{o}f \textbf{R}easoning with \textbf{G}rounded \textbf{I}nsights), a framework that enhances reasoning reliability through post-hoc verification of chain-of-thought outputs. Given a VLM-generated rationale, CoRGI decomposes it into step-wise statements, grounds each step in visual evidence, and filters or corrects unsupported claims before producing the final answer. Experiments on five challenging benchmark-VCR, ScienceQA, MMMU, MathVista, and HallusionBenc-demonstrate that CoRGI consistently improves both answer accuracy and explanation faithfulness across multiple VLM backbones, including Qwen-2.5VL, LLaVA-1.6, and Gemma3-12B. Beyond quantitative gains, qualitative analyses further illustrate how the verification process reduces hallucination and strengthens interpretability, suggesting that post-hoc visual grounding is a promising direction for building more trustworthy and transparent multimodal reasoning systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑