视频生成器即机器人策略
Video Generators are Robot Policies
AI总结：
针对现有视觉运动策略泛化性差、受演示数据规模限制的问题，提出融合视频与动作生成的可端到端训练的Video Policy框架，以视频生成为代理学习策略，大幅提升样本效率与泛化能力，性能优于传统行为克隆。
摘要：
尽管灵巧操作领域已取得巨大进展，当前的视觉运动策略仍从根本上受限于两大挑战：它们难以在感知或行为分布偏移下实现泛化，且性能受人类演示数据规模的制约。本文中，我们将视频生成作为机器人策略学习的代理，以同时解决这两项局限。我们提出Video Policy（视频策略），这是一个融合视频生成与动作生成的模块化框架，可实现端到端训练。我们的研究结果表明，学习生成机器人行为的视频，能够以极少的演示数据提取出策略，显著提升了鲁棒性和样本效率。该方法在模拟环境和真实世界中，均对未见过的物体、背景和任务展现出强泛化能力。我们进一步指出，任务成功率与生成的视频密切相关，无动作的视频数据为泛化到新任务提供了关键助力。通过利用大规模视频生成模型，我们的方法相比传统行为克隆取得了更优性能，为更具可扩展性、数据效率更高的机器人策略学习铺平了道路。
英文摘要：
Despite tremendous progress in dexterous manipulation, current visuomotor policies remain fundamentally limited by two challenges: they struggle to generalize under perceptual or behavioral distribution shifts, and their performance is constrained by the size of human demonstration data. In this paper, we use video generation as a proxy for robot policy learning to address both limitations simultaneously. We propose Video Policy, a modular framework that combines video and action generation that can be trained end-to-end. Our results demonstrate that learning to generate videos of robot behavior allows for the extraction of policies with minimal demonstration data, significantly improving robustness and sample efficiency. Our method shows strong generalization to unseen objects, backgrounds, and tasks, both in simulation and the real world. We further highlight that task success is closely tied to the generated video, with action-free video data providing critical benefits for generalizing to novel tasks. By leveraging large-scale video generative models, we achieve superior performance compared to traditional behavior cloning, paving the way for more scalable and data-efficient robot policy learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
