RoboScape-R: 通用机器人训练的统一奖励-观测世界模型
RoboScape-R: Unified Reward-Observation World Models for Generalizable Robotics Training via RL
AI总结：
RoboScape-R通过统一奖励-观测世界模型提升机器人训练的泛化能力，实现37.5%的性能提升。
摘要：
实现可泛化的具身策略仍然是一个关键挑战。传统策略学习范式，包括模仿学习（IL）和强化学习（RL），在多样场景中培养泛化能力方面存在困难。虽然IL策略往往过拟合特定专家轨迹，RL则因缺乏统一且通用的奖励信号而难以实现多场景泛化。我们提出世界模型能够作为通用环境代理来解决这一限制。然而，当前的世界模型主要关注其预测观测的能力，仍依赖任务特定的手工奖励函数，因此无法提供真正通用的训练环境。为此，我们提出RoboScape-R框架，利用世界模型作为RL范式中具身环境的通用代理。我们引入了一种基于世界模型的通用奖励机制，生成源于模型内在理解的现实世界状态转换动态的''内生''奖励。大量实验表明，RoboScape-R有效解决了传统RL方法的局限性，通过提供高效且通用的训练环境，显著增强了具身策略的泛化能力。我们的方法为利用世界模型作为在线训练策略提供了关键见解，并在跨领域场景中实现了平均37.5%的性能提升。
英文摘要：
Achieving generalizable embodied policies remains a key challenge. Traditional policy learning paradigms, including both Imitation Learning (IL) and Reinforcement Learning (RL), struggle to cultivate generalizability across diverse scenarios. While IL policies often overfit to specific expert trajectories, RL suffers from the inherent lack of a unified and general reward signal necessary for effective multi-scene generalization. We posit that the world model is uniquely capable of serving as a universal environment proxy to address this limitation. However, current world models primarily focus on their ability to predict observations and still rely on task-specific, handcrafted reward functions, thereby failing to provide a truly general training environment. Toward this problem, we propose RoboScape-R, a framework leveraging the world model to serve as a versatile, general-purpose proxy for the embodied environment within the RL paradigm. We introduce a novel world model-based general reward mechanism that generates ''endogenous'' rewards derived from the model's intrinsic understanding of real-world state transition dynamics. Extensive experiments demonstrate that RoboScape-R effectively addresses the limitations of traditional RL methods by providing an efficient and general training environment that substantially enhances the generalization capability of embodied policies. Our approach offers critical insights into utilizing the world model as an online training strategy and achieves an average 37.5% performance improvement over baselines under out-of-domain scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
