Video2Policy：通过互联网视频在仿真中扩展操作任务
Video2Policy: Scaling up Manipulation Tasks in Simulation through Internet Videos
AI总结：
本文提出 Video2Policy 框架，从互联网 RGB 视频重建仿真操作任务，并结合 LLM 迭代生成奖励进行强化学习，实现百余个视频任务训练及 Real2Sim2Real 迁移。
摘要：
仿真为低成本扩展通用策略的训练数据提供了一种有前景的方法。为了从多样且真实的任务中可扩展地生成数据，现有算法要么依赖大型语言模型（LLM），而这些模型可能会幻觉出对机器人学没有意义的任务；要么依赖数字孪生，而数字孪生需要精细的现实到仿真对齐，并且难以扩展。为应对这些挑战，我们提出了 Video2Policy，这是一个利用互联网 RGB 视频、基于人类日常行为重建任务的新颖框架。我们的方法包括两个阶段：（1）从视频中在仿真里生成任务；（2）利用上下文中由 LLM 迭代生成的奖励函数进行强化学习。我们通过重建 Something-Something-v2（SSv2）数据集中的 100 多个视频展示了 Video2Policy 的有效性，这些视频描绘了 9 个不同任务上多样且复杂的人类行为。我们的方法能够在此类任务上成功训练 RL 策略，包括投掷等复杂且具有挑战性的任务。最后，我们表明生成的仿真数据可以扩展用于训练通用策略，并且能够以 Real2Sim2Real 的方式迁移回真实机器人。
英文摘要：
Simulation offers a promising approach for cheaply scaling training data for generalist policies. To scalably generate data from diverse and realistic tasks, existing algorithms either rely on large language models (LLMs) that may hallucinate tasks not interesting for robotics; or digital twins, which require careful real-to-sim alignment and are hard to scale. To address these challenges, we introduce Video2Policy, a novel framework that leverages internet RGB videos to reconstruct tasks based on everyday human behavior. Our approach comprises two phases: (1) task generation in simulation from videos; and (2) reinforcement learning utilizing in-context LLM-generated reward functions iteratively. We demonstrate the efficacy of Video2Policy by reconstructing over 100 videos from the Something-Something-v2 (SSv2) dataset, which depicts diverse and complex human behaviors on 9 different tasks. Our method can successfully train RL policies on such tasks, including complex and challenging tasks such as throwing. Finally, we show that the generated simulation data can be scaled up for training a general policy, and it can be transferred back to the real robot in a Real2Sim2Real way.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
