Embodied-R：通过强化学习激活基础模型中具身空间推理的协同框架
Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning
AI总结：
针对预训练模型获取具身空间推理能力的问题，提出结合VLMs感知与小规模LMs推理的Embodied-R协同框架，通过RL及新奖励系统实现低资源慢思考，仅用5k样本训练即匹配SOTA模型性能并展现涌现思维模式。
摘要：
人类能够从顺序视觉观察（如自我中心视频流）中感知并推理空间关系。然而，预训练模型如何获取此类能力，尤其是高级推理能力，仍不明确。本文介绍Embodied-R，一个结合用于感知的大规模视觉语言模型（VLMs）和用于推理的小规模语言模型（LMs）的协同框架。该模型采用强化学习（RL）和考虑思考-答案逻辑一致性的新型奖励系统，在有限计算资源下实现慢思考能力。仅在5k个具身视频样本上训练后，搭载3B参数LM的Embodied-R在分布内和分布外的具身空间推理任务中，性能均匹配OpenAI-o1、Gemini-2.5-pro等最先进多模态推理模型。Embodied-R还展现出系统分析和上下文整合等涌现思维模式。我们进一步探索了响应长度、VLM训练、奖励设计策略以及监督微调（SFT）与RL训练后模型泛化差异等研究问题。
英文摘要：
Humans can perceive and reason about spatial relationships from sequential visual observations, such as egocentric video streams. However, how pretrained models acquire such abilities, especially high-level reasoning, remains unclear. This paper introduces Embodied-R, a collaborative framework combining large-scale Vision-Language Models (VLMs) for perception and small-scale Language Models (LMs) for reasoning. Using Reinforcement Learning (RL) with a novel reward system considering think-answer logical consistency, the model achieves slow-thinking capabilities with limited computational resources. After training on only 5k embodied video samples, Embodied-R with a 3B LM matches state-of-the-art multimodal reasoning models (OpenAI-o1, Gemini-2.5-pro) on both in-distribution and out-of-distribution embodied spatial reasoning tasks. Embodied-R also exhibits emergent thinking patterns such as systematic analysis and contextual integration. We further explore research questions including response length, training on VLM, strategies for reward design, and differences in model generalization after SFT (Supervised Fine-Tuning) and RL training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
