Grounded-Dreamer：基于物理 grounding 的世界模型合成数据进行机器人学习
Grounded-Dreamer: Robot Learning from World Model Synthetic Data with Physical Grounding
AI总结：
Grounded-Dreamer框架结合生成式视频模型与仿真物理引擎，从少量真实图像合成带物理 grounding 的机器人操纵数据，无需人类远程操作，可高效训练机器人操纵策略，实验验证其效果显著。
摘要：
扩展机器人操纵策略需要大量多样化的带动作标签的数据，但在现实世界中收集此类数据成本高昂且难以扩展。现有方法要么依赖人类远程操作，要么依赖视觉真实度有限的仿真环境，要么使用缺乏特定实体注释的人类视频。本文提出了Grounded-Dreamer，这是一种利用视频世界模型合成带物理 grounding 的机器人操纵数据的框架，同时避免人类远程操作演示。Grounded-Dreamer结合了生成式视频模型的视觉真实度与仿真器物理引擎的优势。从少量描述机器人任务初始状态的真实图像出发，该方法自动重构任务相关对象与相机几何结构，将其在仿真环境中对齐，并收集物理上有效的轨迹。这些轨迹被用于后训练视频世界模型与逆动力学模型，从而能够从真实图像生成视觉真实的机器人操纵视频及对应的机器人动作。为过滤物理上不合理的 rollout，Grounded-Dreamer会在重构的仿真环境中回放生成的轨迹，仅保留成功完成任务的轨迹。现实世界机器人策略仅基于带物理 grounding 的合成轨迹进行训练，无需任何人类演示。现实世界任务的实验结果表明，Grounded-Dreamer能够合成带物理 grounding 的数据，且无需人类远程操作即可大幅促进策略学习。
英文摘要：
Scaling robot manipulation policies requires large amounts of diverse, action-labeled data, yet collecting such data in the real world remains expensive and difficult to scale. Existing methods either depend on human teleoperation, rely on simulation with limited visual realism, or use human videos that lack embodiment-specific annotations. In this paper, we introduce Grounded-Dreamer, a framework for synthesizing physically grounded robot manipulation data with video world models while avoiding human teleoperation demonstrations. Grounded-Dreamer combines the visual realism of generative video models with simulator physics engines. Starting from a small set of real images depicting robot task initial states, our method automatically reconstructs task-relevant objects and camera geometry, aligns them in simulation, and collects physically valid trajectories. These trajectories are used to post-train both a video world model and an inverse dynamics model, enabling the generation of visually realistic robot manipulation videos and corresponding robot actions from real images. To filter physically implausible rollouts, Grounded-Dreamer replays generated trajectories in the reconstructed simulation and retains only those that successfully complete the tasks. The real-world robot policies are trained solely on the grounded synthetic trajectories without any human demonstrations. Empirical results in real-world tasks demonstrate that Grounded-Dreamer can synthesize physically grounded data and substantially facilitate policy learning without requiring human teleoperation effort.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
