Motion Tracks：少样本模仿学习中人机迁移的统一表示
Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learning
AI总结：
本文提出将动作表示为图像上的短时程2D轨迹（motion tracks），实现跨本体的人机迁移，仅需少量人类视频和机器人演示即可训练策略，在4个真实任务中达到86.5%的成功率。
摘要：
教会机器人自主完成日常任务仍然是一个挑战。模仿学习（Imitation Learning, IL）是一种强大的方法，通过示教赋予机器人技能，但受限于收集遥操作机器人数据的劳动密集型过程。人类视频提供了一种可扩展的替代方案，但由于缺乏机器人动作标签，直接从中训练IL策略仍然困难。为了解决这个问题，我们提出将动作表示为图像上的短时程2D轨迹。这些动作（或称motion tracks）捕捉了人手或机器人末端执行器的预测运动方向。我们实例化了一个名为Motion Track Policy（MT-pi）的IL策略，它接收图像观测并输出motion tracks作为动作。通过利用这种统一的跨本体动作空间，MT-pi仅需几分钟的人类视频和有限的额外机器人演示即可高成功率地完成任务。在测试时，我们从两个相机视角预测motion tracks，通过多视图合成恢复6DoF轨迹。MT-pi在4个真实世界任务上实现了86.5%的平均成功率，比不利用人类数据或我们动作空间的最先进IL基线高出40%，并能泛化到仅在人类视频中见过的场景。代码和视频可在我们的网站https://portal-cornell.github.io/motion_track_policy/获取。
英文摘要：
Teaching robots to autonomously complete everyday tasks remains a challenge. Imitation Learning (IL) is a powerful approach that imbues robots with skills via demonstrations, but is limited by the labor-intensive process of collecting teleoperated robot data. Human videos offer a scalable alternative, but it remains difficult to directly train IL policies from them due to the lack of robot action labels. To address this, we propose to represent actions as short-horizon 2D trajectories on an image. These actions, or motion tracks, capture the predicted direction of motion for either human hands or robot end-effectors. We instantiate an IL policy called Motion Track Policy (MT-pi) which receives image observations and outputs motion tracks as actions. By leveraging this unified, cross-embodiment action space, MT-pi completes tasks with high success given just minutes of human video and limited additional robot demonstrations. At test time, we predict motion tracks from two camera views, recovering 6DoF trajectories via multi-view synthesis. MT-pi achieves an average success rate of 86.5% across 4 real-world tasks, outperforming state-of-the-art IL baselines which do not leverage human data or our action space by 40%, and generalizes to scenarios seen only in human videos. Code and videos are available on our website https://portal-cornell.github.io/motion_track_policy/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
