SVoT: 基于强化学习的空间推理状态感知思维可视化
SVoT: State-aware Visualization-of-Thought for Spatial Reasoning via Reinforcement Learning
AI总结：
提出SVoT框架，通过强化学习生成可验证的中间状态和可视化，结合文本与视觉推理链，提升多模态大模型在多跳空间推理中的可靠性。
摘要：
空间推理对多模态大语言模型（MLLMs）仍是一个挑战，因为它需要在中间状态和状态转换上进行可靠的多跳推理。当前研究通常不验证中间状态，并将状态转换视为隐式过程，这限制了多跳空间推理的可靠性。为解决这一问题，我们提出状态感知思维可视化（SVoT），一种强化学习框架，生成交错、可验证的中间状态和可视化。SVoT将转换推理链整合到生成过程中，使模型能够通过交错的文本和视觉推理验证动作前提和效果。我们通过组相对策略优化（GRPO）训练SVoT，通过奖励设计实例化验证，并评估不同细粒度奖励的效果。由于现有基准将状态转换简化为单变量更新，大大简化了问题，我们通过扩展经典环境并引入两个需要多对象交互和数值推理的新领域Pacman和Gather，建立了五个领域。这些领域支持对多跳空间推理的系统评估，并对生成的中间状态和转换推理进行定量验证。具有转换感知监督的SVoT在引入的领域中达到了最先进的性能，在分布外测试集上实现了高达65%的绝对准确率提升。
英文摘要：
Spatial reasoning remains a challenge for Multimodal Large Language Models (MLLMs), as it requires reliable multi-hop inference over both intermediate states and state transitions. Current studies often leave intermediate states unverified and treat state transitions as implicit processes, which limits reliability in multi-hop spatial reasoning. To address this, we propose State-aware Visualization-of-Thought (SVoT), a reinforcement learning framework that generates interleaved, verifiable intermediate states and visualizations. SVoT integrates transition reasoning chains into the generation processes, enabling the model to verify action preconditions and effects through interleaved textual and visual reasoning. We train SVoT via Group Relative Policy Optimization (GRPO), instantiating verification through reward design and evaluating the efficacy of different fine-grained rewards. As existing benchmarks reduce state transitions to single-variable updates, substantially simplifying the problems, we establish five domains by extending classical environments and introducing two novel domains, Pacman and Gather, that require multi-object interactions and numerical reasoning. These domains support systematic evaluation of multi-hop spatial reasoning with quantitative verification of generated intermediate states and transition reasoning. SVoT with transition-aware supervision achieves state-of-the-art performance across the introduced domains, yielding up to a 65% absolute accuracy gain on out-of-distribution test sets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
