iVGR: 通过强化学习将视觉基础推理内化到多模态大语言模型中
iVGR: Internalizing Visually Grounded Reasoning for MLLMs with Reinforcement Learning
AI总结：
提出iVGR框架，利用强化学习和双流训练策略将视觉定位能力内化到文本推理中，避免显式视觉基础在推理时的干扰，提升细粒度感知性能。
摘要：
尽管视觉基础链式思维（CoT）已成为增强多模态大语言模型（MLLM）细粒度感知的有前途范式，但其在推理阶段的有效性仍未得到充分探索。在这项工作中，我们经验性地发现，与没有显式视觉基础的标准文本CoT相比，在推理时强制要求视觉基础CoT中的显式对象框通常会降低性能。我们假设视觉定位能力可以内化到文本CoT中，而强制性的显式基础会对模型的主要目标（答案预测）引入不必要的干扰。为了解决这个问题，我们提出了内化视觉基础推理（iVGR），一种新颖的强化学习框架，将定位能力转移到文本推理过程中。我们采用双流训练策略，通过提出的一致性奖励将文本流与高质量的视觉基础流对齐，使模型在推理时无需显式基础即可准确定位。大量实验表明，我们的方法在细粒度基准上显著优于现有基线，同时保持支持工具辅助推理工作流的灵活性。
英文摘要：
While visually grounded Chain-of-Thought (CoT) has emerged as a promising paradigm to enhance fine-grained perception in multimodal large language models (MLLMs), its efficacy during the inference phase remains underexplored. In this work, we empirically find that mandating explicit object boxes in visually grounded CoT during inference often degrades performance compared to standard textual CoT, which reasons without explicit visual grounding. We hypothesize that the visual localization capability can be internalized into the textual CoT and that the mandatory explicit grounding introduces unnecessary interference with the model's primary objective of answer prediction. To address this problem, we propose Internalizing Visually Grounded Reasoning (\textbf{iVGR}), a novel reinforcement learning framework that transfers localization capabilities into the textual reasoning process. We employ a dual-stream training strategy, where a textual stream is aligned with a high-quality visually grounded stream via a proposed consistency reward, enabling the model to localize accurately without explicit grounding during inference. Extensive experiments demonstrate that our method significantly outperforms existing baselines on fine-grained benchmarks, while maintaining the flexibility to support tool-assisted inference workflows.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
