GE-Sim 2.0:迈向机器人操作综合闭环视频世界模拟器的路线图
GE-Sim 2.0: A Roadmap Towards Comprehensive Closed-loop Video World Simulators for Robotic Manipulation
AI总结:
提出GE-Sim 2.0,一种基于动作条件视频生成的闭环视频世界模拟器,通过重新训练数千小时真实机器人数据并新增状态专家、世界裁判和加速框架三个模块,实现高保真动作跟随和轨迹覆盖,在WorldArena排行榜上以2B参数超越专用模型和通用视频生成器,并验证了基于其生成轨迹和奖励训练的策略在真实世界中的有效性。
摘要:
我们介绍了GE-Sim 2.0(Genie Envisioner世界模拟器2.0),一种用于机器人操作的闭环视频世界模拟器。基于Genie Envisioner的动作条件视频生成框架,GE-Sim 2.0在数千小时的真实机器人数据上重新训练,涵盖遥操作、接触丰富交互和机载策略部署,显著提高了动作跟随保真度和轨迹覆盖范围。在此基础之上,三个新模块实现了从视频模拟到策略学习的闭环:一个状态专家,从视频潜在表示中解码本体感觉状态,以支持下游VLA策略的下一块预测;一个世界裁判,根据任务指令对生成的轨迹进行评分,提供机器可验证的成功信号和奖励,取代人工检查;以及一个加速框架,在单个H100上以2.3秒生成25帧轨迹,并在推理时支持高达4倍跳帧以实现长程评估。GE-Sim 2.0仅以2B参数便登顶公开的WorldArena排行榜,超越了专用机器人世界模型和闭源通用视频生成器,并且基于其生成轨迹和奖励训练的策略可转化为可测量的真实世界收益,确立了GE-Sim 2.0作为可扩展评估和操作策略闭环学习的实用平台。
英文摘要:
We introduce GE-Sim 2.0 (Genie Envisioner World Simulator 2.0), a closed-loop video world simulator for robotic manipulation. Building on the action-conditioned video generation framework of Genie Envisioner, GE-Sim 2.0 is re-trained on thousands of hours of real-world robot data spanning teleoperation, contact-rich interaction, and on-robot policy deployment, substantially improving action-following fidelity and trajectory coverage. On top of this foundation, three new modules close the loop from video simulation to policy learning: a state expert that decodes proprioceptive state from video latents to support next-chunk prediction by downstream VLA policies; a world judge that scores generated rollouts against task instructions, yielding machine-verifiable success signals and rewards in place of manual inspection; and an acceleration framework that delivers a 25-frame rollout in 2.3 seconds on a single H100, with up to 4* frame skipping at inference for long-horizon evaluation. GE-Sim 2.0 tops the public WorldArena leaderboard at only 2B parameters, outperforming both dedicated robotic world models and closed-source general video generators, and policies trained against its rollouts and rewards translate into measurable real-world gains, establishing GE-Sim 2.0 as a practical platform for scalable evaluation and closed-loop learning of manipulation policies.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。