助机器人一臂之力：利用眼在手上的人类视频演示学习可泛化的操作
Giving Robots a Hand: Learning Generalizable Manipulation with Eye-in-Hand Human Video Demonstrations
AI总结：
本文提出用未标注的眼在手上人类视频扩充机器人模仿数据，并借助部分可观测性与固定图像掩码缩小域差距，在8项真实任务上平均绝对提升成功率58%。
摘要：
眼在手上相机已展现出在基于视觉的机器人操作中实现更高样本效率和泛化能力的前景。然而，对于机器人模仿而言，让人类遥操作员使用真实机器人收集大量专家演示仍然成本高昂。相比之下，人类执行任务的视频收集成本要低得多，因为这些视频无需机器人遥操作方面的专业知识，并且能够在广泛场景中快速采集。因此，人类视频演示是大规模学习可泛化机器人操作策略的一种有前景的数据来源。在本研究中，我们使用广泛的、未标注的人类视频演示来扩充范围狭窄的机器人模仿数据集，从而显著增强眼在手上视觉运动策略的泛化能力。尽管人类数据与机器人数据之间存在明显的视觉域差距，但我们的框架不需要采用任何显式域适应方法，因为我们利用了眼在手上相机的部分可观测性以及一种简单的固定图像掩码方案。在包含3-DoF和6-DoF机械臂控制的八项真实世界任务套件上，我们的方法使眼在手上操作策略的成功率平均提高了58%（绝对值），使机器人能够泛化到机器人演示数据中未见过的新环境配置和新任务。视频结果请参见 https://giving-robots-a-hand.github.io/ 。
英文摘要：
Eye-in-hand cameras have shown promise in enabling greater sample efficiency and generalization in vision-based robotic manipulation. However, for robotic imitation, it is still expensive to have a human teleoperator collect large amounts of expert demonstrations with a real robot. Videos of humans performing tasks, on the other hand, are much cheaper to collect since they eliminate the need for expertise in robotic teleoperation and can be quickly captured in a wide range of scenarios. Therefore, human video demonstrations are a promising data source for learning generalizable robotic manipulation policies at scale. In this work, we augment narrow robotic imitation datasets with broad unlabeled human video demonstrations to greatly enhance the generalization of eye-in-hand visuomotor policies. Although a clear visual domain gap exists between human and robot data, our framework does not need to employ any explicit domain adaptation method, as we leverage the partial observability of eye-in-hand cameras as well as a simple fixed image masking scheme. On a suite of eight real-world tasks involving both 3-DoF and 6-DoF robot arm control, our method improves the success rates of eye-in-hand manipulation policies by 58% (absolute) on average, enabling robots to generalize to both new environment configurations and new tasks that are unseen in the robot demonstration data. See video results at https://giving-robots-a-hand.github.io/ .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
