提问的艺术：一致性增强空间推理中的事实性
The Art of Interrogation: Consistency Amplifies Factuality in Spatial Reasoning
AI总结：
提出自监督强化学习框架，通过几何与语义一致性验证器（如图像翻转、文本对象顺序交换）对齐预训练模型的内在空间推理能力，无需标注数据即可达到接近监督方法的精度。
摘要：
当前的大型推理模型（LRMs）展现出显著的通用能力，但在空间推理任务中表现明显不足。现有方法将此差距视为知识缺陷，依赖监督微调（SFT）从外部视觉源或合成引擎中获取标注空间数据。相反，我们认为对于许多任务，空间推理能力已经存在于预训练的LRMs中，但需要通过几何2D和3D约束下的逻辑一致性进行对齐。在这项工作中，我们提出了一个自监督强化学习（RL）框架，针对内部推理过程，无需真实标注。通过形式化一致性验证器——即在变换下检查几何和语义一致性的奖励函数——我们证明模型可以提高其空间推理能力。我们同时使用图像变换（如翻转）和文本变换（如交换问题中对象的顺序），并提出了一种新的基于最优传输的RL策略OT-GRPO，这是针对成对验证器定制的组相对策略优化的最小匹配变体。我们展示了这种无标签一致性训练在精度上接近使用真实监督训练的模型，并在不同任务和数据领域实现了类似的泛化。
英文摘要：
Current Large Reasoning Models (LRMs) exhibit remarkable general capabilities but significantly underperform in spatial reasoning tasks. Existing approaches treat this gap as a knowledge deficit, relying on supervised fine-tuning (SFT) to ingest labeled spatial data from external vision sources or synthetic engines. In contrast, we argue that for many tasks, spatial reasoning capabilities are already present in pre-trained LRMs but require alignment through logical coherence under geometric 2D and 3D constraints. In this work, we propose a self-supervised reinforcement learning (RL) framework that targets the internal reasoning process without requiring ground-truth annotations. By formalizing the notion of consistency verifiers -- reward functions that check for geometric and semantic consistency under transformations -- we demonstrate that models can improve their spatial reasoning abilities. We use both image transformations, like flipping, and textual transformations, like swapping the order of objects in the question, and propose a new optimal transport-based RL strategy, OT-GRPO, which is a minimal-matching variant of group relative policy optimization tailored to pairwise verifiers. We show that this label-free consistency training approaches the accuracy of models trained with ground-truth supervision and achieves similar generalization across diverse tasks and data domains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
