VG-CoT：通过 grounded Chain-of-Thought 实现可信的视觉推理
VG-CoT: Towards Trustworthy Visual Reasoning via Grounded Chain-of-Thought
AI总结：
本文提出 VG-CoT 数据集，通过自动化三阶段流程将推理步骤与图像真实视觉证据联系起来，提升大视觉-语言模型的可信度和推理能力。
摘要：
大视觉-语言模型（LVLMs）的发展需要精确的局部区域推理，能够真实地将模型逻辑 grounded 在实际视觉证据中。然而，现有数据集由于大量手动标注和多步推理与相应图像区域之间缺乏显式对齐，限制了模型可信度的评估。为解决这些问题，我们提出了视觉 grounded Chain-of-Thought（VG-CoT）数据集，通过完全自动化的三阶段流程，将每个推理步骤与图像中的真实视觉证据联系起来。流程首先使用最先进的检测和OCR模型提取对象和文本级别的视觉证据，然后通过GPT-4o生成逐步 grounded 推理，最后通过基于推理的开放集检测过程进行优化。此外，我们引入了一个新的基准，全面评估LVLMs在三个互补维度上的推理能力：推理质量、答案准确性以及推理-答案对齐度。使用代表性LVLMs，包括LLaVA-1.5和Qwen2-VL，实验显示在大多数评估指标上都有显著改进，证明VG-CoT有效提升了基于证据的可信推理，同时保持了可扩展性和成本效益的数据集构建。数据集和代码将在接受后公开发布，以促进进一步研究。
英文摘要：
The advancement of Large Vision-Language Models (LVLMs) requires precise local region-based reasoning that faithfully grounds the model's logic in actual visual evidence. However, existing datasets face limitations in scalability due to extensive manual annotation and lack of explicit alignment between multi-step reasoning and corresponding image regions, which constrains the evaluation of model trustworthiness. To address these challenges, we propose the Visual Grounding Chain-of-Thought (VG-CoT) dataset, which explicitly links each reasoning step to real visual evidence within the image through a fully automated three-stage pipeline. The pipeline first extracts object- and text-level visual evidence using state-of-the-art detection and OCR models, then generates step-by-step grounded reasoning with GPT-4o, and finally refines the grounding through a rationale-driven open-set detection process. In addition, we introduce a new benchmark that comprehensively evaluates LVLMs reasoning across three complementary dimensions: Rationale Quality, Answer Accuracy, and Reasoning-Answer Alignment. Experiments with representative LVLMs, including LLaVA-1.5 and Qwen2-VL, demonstrate consistent improvements on most evaluation metrics, confirming that VG-CoT effectively enhances trustworthy, evidence-based reasoning while maintaining scalable and cost-efficient dataset construction. The dataset and code will be released publicly upon acceptance to facilitate further research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
