VGR：视觉基础推理
VGR: Visual Grounded Reasoning
AI总结：
本文提出VGR，一种增强视觉感知的多模态大语言模型，通过图像区域检测与回放提升多模态推理能力，在多个基准测试中表现优异。
摘要：
在多模态链式推理领域，现有方法主要依赖纯语言空间推理，存在语言偏差且局限于数学或科学领域。为解决这些问题，本文引入VGR，一种具有增强细粒度视觉感知能力的新型多模态大语言模型。与传统MLLM不同，VGR首先检测有助于解决问题的相关区域，然后基于回放的图像区域提供精确答案。为此，我们构建了一个大规模SFT数据集VGR-SFT，包含混合视觉基础和语言推断的推理数据。VGR的推理流程允许模型选择视觉参考的边界框，并引入回放阶段将相应区域整合到推理过程中，提升多模型理解能力。实验表明，VGR在需要全面图像细节理解的多模态基准测试中表现优异。与基线相比，VGR仅使用30%的图像token数量，但在MMStar上提高4.1分，在AI2D上提高7.1分，在ChartQA上提高12.9分。
英文摘要：
In the field of multimodal chain-of-thought (CoT) reasoning, existing approaches predominantly rely on reasoning on pure language space, which inherently suffers from language bias and is largely confined to math or science domains. This narrow focus limits their ability to handle complex visual reasoning tasks that demand comprehensive understanding of image details. To address these limitations, this paper introduces VGR, a novel reasoning multimodal large language model (MLLM) with enhanced fine-grained visual perception capabilities. Unlike traditional MLLMs that answer the question or reasoning solely on the language space, our VGR first detects relevant regions that may help to solve problems, and then provides precise answers based on replayed image regions. To achieve this, we conduct a large-scale SFT dataset called VGR -SFT that contains reasoning data with mixed vision grounding and language deduction. The inference pipeline of VGR allows the model to choose bounding boxes for visual reference and a replay stage is introduced to integrates the corresponding regions into the reasoning process, enhancing multimodel comprehension. Experiments on the LLaVA-NeXT-7B baseline show that VGR achieves superior performance on multi-modal benchmarks requiring comprehensive image detail understanding. Compared to the baseline, VGR uses only 30\% of the image token count while delivering scores of +4.1 on MMStar, +7.1 on AI2D, and a +12.9 improvement on ChartQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
