解析-增强-蒸馏：从单个人类视频学习可泛化的双臂视觉运动策略
Parse-Augment-Distill: Learning Generalizable Bimanual Visuomotor Policies from Single Human Video
AI总结：
提出PAD框架，从单个人类视频解析关键点轨迹，经双臂任务规划增强后蒸馏为策略，实现高效可泛化的双臂操作。
摘要：
从专家演示中学习视觉运动策略是现代机器人研究的重要前沿方向，然而，大多数流行方法需要耗费大量精力收集遥操作数据，并且难以泛化到分布外场景。通过利用人类视频以及演示增强技术，数据收集的规模化已得到探索。后者方法通常需要昂贵的仿真回滚，并使用合成图像数据训练策略，因此引入了仿真到现实的差距。与此同时，替代性状态表示（如关键点）在类别级泛化方面展现出巨大潜力。在这项工作中，我们将这些途径整合到一个统一框架中：PAD（解析-增强-蒸馏），用于从单个人类视频学习可泛化的双臂策略。我们的方法依赖三个步骤：（a）将人类视频演示解析为机器人可执行的关键点-动作轨迹，（b）采用双臂任务与运动规划，在无需仿真器的情况下大规模增强演示，（c）将增强后的轨迹蒸馏为关键点条件策略。实验上，我们展示了PAD在成功率以及样本/成本效率方面，均优于依赖图像策略和仿真回滚的最先进双臂演示增强方法。我们在六个多样化的真实世界双臂任务中部署了我们的框架，例如倒饮料、清理垃圾和打开容器，产生了一次性策略，这些策略能够泛化到未见过的空间布局、物体实例和背景干扰物。补充材料可在项目网页https://gtziafas.github.io/PAD_project/找到。
英文摘要：
Learning visuomotor policies from expert demonstrations is an important frontier in modern robotics research, however, most popular methods require copious efforts for collecting teleoperation data and struggle to generalize out-ofdistribution. Scaling data collection has been explored through leveraging human videos, as well as demonstration augmentation techniques. The latter approach typically requires expensive simulation rollouts and trains policies with synthetic image data, therefore introducing a sim-to-real gap. In parallel, alternative state representations such as keypoints have shown great promise for category-level generalization. In this work, we bring these avenues together in a unified framework: PAD (Parse-AugmentDistill), for learning generalizable bimanual policies from a single human video. Our method relies on three steps: (a) parsing a human video demo into a robot-executable keypoint-action trajectory, (b) employing bimanual task-and-motion-planning to augment the demonstration at scale without simulators, and (c) distilling the augmented trajectories into a keypoint-conditioned policy. Empirically, we showcase that PAD outperforms state-ofthe-art bimanual demonstration augmentation works relying on image policies with simulation rollouts, both in terms of success rate and sample/cost efficiency. We deploy our framework in six diverse real-world bimanual tasks such as pouring drinks, cleaning trash and opening containers, producing one-shot policies that generalize in unseen spatial arrangements, object instances and background distractors. Supplementary material can be found in the project webpage https://gtziafas.github.io/PAD_project/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
