视觉锚定思维
Thinking with Visual Grounding
AI总结：
提出视觉锚定思维方法，让视觉语言模型在推理时交替生成自然语言和视觉锚点（点或框），并通过合成数据管道和锚定感知强化学习训练，在计数和空间推理任务上显著提升性能。
摘要：
视觉思维不仅应该听起来正确，还应该展示其证据。虽然最近的视觉语言模型（VLM）能够生成自然语言推理轨迹，但这些轨迹往往隐含了所支持的图像区域，使得它们难以验证和监督。我们引入了视觉锚定思维，这是一种推理过程，其中模型将自然语言思想与每一步所使用的视觉证据的显式点或框锚定交替生成。这使得模型能够在语言中表达中间推理，同时将关键对象锚定到它们所指的图像区域。为了训练这种行为，我们构建了一个可扩展的合成管道，该管道蒸馏正确的视觉推理轨迹，提取轨迹所需的视觉对象，使用基于SAM3的代理对其进行锚定，并从生成的掩码中导出对齐的点与框监督。我们进一步提出了锚定感知强化学习，它将答案正确性奖励与密集的锚定奖励相结合，后者评分生成的物体引用是否匹配正确的图像证据。在两个计数基准和四个空间推理基准上，将视觉锚定思维添加到Gemma3-4B-IT中，始终优于原始模型和非锚定思维基线。在空间推理上，视觉锚定思维的4B模型匹配，并在某些情况下超越了同一模型家族的Gemma3-27B-IT。我们的分析表明，点锚定适合计数，而框锚定在空间任务上从显式锚定奖励中获益最多。总体而言，我们的结果表明，当VLM的中间思维与使它们为真的图像区域相关联时，它们的思考能力更强。
英文摘要：
Visual thinking should not only sound right; it should show its evidence. While recent vision-language models (VLMs) can produce natural-language reasoning traces, these traces often leave the supporting image regions implicit, making them hard to verify and difficult to supervise. We introduce visually grounded thinking, a reasoning process in which models interleave natural-language thoughts with explicit point or box groundings of the visual evidence used at each step. This lets the model express intermediate reasoning in language while grounding key objects in the image regions they refer to. To train this behavior, we construct a scalable synthesis pipeline that distills correct visual reasoning traces, extracts the visual objects required by the traces, grounds them with a SAM3-based agent, and derives aligned point and box supervision from the resulting masks. We further propose grounding-aware reinforcement learning, which combines answer correctness rewards with dense grounding rewards that score whether generated object references match the correct image evidence. Across two counting benchmarks and four spatial reasoning benchmarks, adding visually grounded thinking to Gemma3-4B-IT consistently improves performance over the original model and the non-grounded thinking baseline. On spatial reasoning, the visually grounded thinking 4B models match, and in some cases surpass, Gemma3-27B-IT from the same model family. Our analysis shows that point grounding is well suited to counting, while box grounding benefits most from explicit grounding rewards on spatial tasks. Overall, our results show that VLMs think better when their intermediate thoughts are tied to the image regions that make them true.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
