DreamGen：通过视频世界模型解锁机器人学习中的泛化能力
DreamGen: Unlocking Generalization in Robot Learning through Video World Models
AI总结：
DreamGen提出一种4阶段流水线，利用视频世界模型生成合成数据训练机器人策略，实现人形机器人22种新行为的环境与行为泛化，并引入DreamGen Bench验证性能。
摘要：
我们引入了DreamGen，这是一个简单但高效的4阶段流水线，用于训练机器人策略，使其通过神经轨迹（从视频世界模型生成的合成机器人数据）在不同行为和环境中实现泛化。DreamGen利用最先进的图像到视频生成模型，将其适配到目标机器人本体，以生成在不同环境中熟悉或新颖任务的照片级逼真合成视频。由于这些模型仅生成视频，我们使用潜在动作模型或逆动力学模型（IDM）恢复伪动作序列。尽管其简单，DreamGen实现了强大的行为和环境泛化：一个人形机器人能够在已见和未见的环境中执行22种新行为，而仅需在一个环境中单个拾取-放置任务的遥操作数据。为了系统评估该流水线，我们引入了DreamGen Bench，一个视频生成基准，显示基准性能与下游策略成功之间的强相关性。我们的工作为扩展机器人学习超越手动数据收集建立了一个有前景的新方向。代码可在https://github.com/NVIDIA/GR00T-Dreams获取。
英文摘要：
We introduce DreamGen, a simple yet highly effective 4-stage pipeline for training robot policies that generalize across behaviors and environments through neural trajectories - synthetic robot data generated from video world models. DreamGen leverages state-of-the-art image-to-video generative models, adapting them to the target robot embodiment to produce photorealistic synthetic videos of familiar or novel tasks in diverse environments. Since these models generate only videos, we recover pseudo-action sequences using either a latent action model or an inverse-dynamics model (IDM). Despite its simplicity, DreamGen unlocks strong behavior and environment generalization: a humanoid robot can perform 22 new behaviors in both seen and unseen environments, while requiring teleoperation data from only a single pick-and-place task in one environment. To evaluate the pipeline systematically, we introduce DreamGen Bench, a video generation benchmark that shows a strong correlation between benchmark performance and downstream policy success. Our work establishes a promising new axis for scaling robot learning well beyond manual data collection. Code available at https://github.com/NVIDIA/GR00T-Dreams.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
