GRIT: 教授大语言模型通过图像思考
GRIT: Teaching MLLMs to Think with Images
AI总结：
GRIT提出一种基于图像和文本的 grounded reasoning 方法，通过强化学习实现高效训练，使大语言模型生成视觉基础的推理链。
摘要：
近期研究表明，使用强化学习（RL）构建能够生成推理链的模型是有效的。然而，尽管在视觉-语言任务中推进推理能力的进展持续，现有开源视觉推理模型通常仅用纯自然语言生成推理内容，缺乏显式整合视觉信息。为此，我们提出了Grounded Reasoning with Images and Texts（GRIT），一种训练大语言模型（MLLMs）通过图像思考的新方法。GRIT引入了一种 grounded reasoning 框架，其中模型生成交替包含自然语言和显式边界框坐标的推理链。这些坐标指向输入图像中模型在推理过程中参考的区域。此外，GRIT配备了基于GRPO算法的强化学习方法GRPO-GR。GRPO-GR采用关注最终答案准确性和接地推理输出格式的鲁棒奖励，从而消除了需要带有推理链标注或显式边界框标签的数据需求。结果表明，GRIT实现了卓越的数据效率，仅需现有数据集中的20个图像-问题-答案三元组。全面评估显示，GRIT有效训练大语言模型生成连贯且视觉基础的推理链，展示了推理和接地能力的成功统一。
英文摘要：
Recent studies have demonstrated the efficacy of using Reinforcement Learning (RL) in building reasoning models that articulate chains of thoughts prior to producing final answers. However, despite ongoing advances that aim at enabling reasoning for vision-language tasks, existing open-source visual reasoning models typically generate reasoning content with pure natural language, lacking explicit integration of visual information. This limits their ability to produce clearly articulated and visually grounded reasoning chains. To this end, we propose Grounded Reasoning with Images and Texts (GRIT), a novel method for training MLLMs to think with images. GRIT introduces a grounded reasoning paradigm, in which models generate reasoning chains that interleave natural language and explicit bounding box coordinates. These coordinates point to regions of the input image that the model consults during its reasoning process. Additionally, GRIT is equipped with a reinforcement learning approach, GRPO-GR, built upon the GRPO algorithm. GRPO-GR employs robust rewards focused on the final answer accuracy and format of the grounded reasoning output, which eliminates the need for data with reasoning chain annotations or explicit bounding box labels. As a result, GRIT achieves exceptional data efficiency, requiring as few as 20 image-question-answer triplets from existing datasets. Comprehensive evaluations demonstrate that GRIT effectively trains MLLMs to produce coherent and visually grounded reasoning chains, showing a successful unification of reasoning and grounding abilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
