RwoR：无需机器人，从人手采集生成机器人示教用于策略学习
RwoR: Generating Robot Demonstrations from Human Hand Collection for Policy Learning without Robot
AI总结：
提出RwoR系统，利用手腕GoPro采集人手示教，通过生成模型转化为机器人夹爪示教，弥合视觉差距，无需机器人即可高效生成训练数据，提升策略学习性能。
摘要：
近年来，模仿学习在机器人操作领域取得了令人瞩目的进展，这得益于高质量训练数据的可得性。为了提高数据采集效率，一些方法专注于开发用于机器人控制的专用遥操作设备，而另一些方法则直接使用人手示教来获取训练数据。然而，前者既需要机器人系统又需要熟练的操作员，限制了其可扩展性；后者则面临人手示教与部署机器人观测之间视觉差距对齐的挑战。为解决这一问题，我们提出了一种人手数据采集系统，并结合我们的手到夹爪生成模型，该模型将人手示教转换为机器人夹爪示教，有效弥合了观测差距。具体而言，我们在人类手腕上安装了一个GoPro鱼眼相机来捕捉人手示教。然后，我们在自采集的成对人手和UMI夹爪示教数据集上训练一个生成模型，该数据集经过定制的数据预处理策略处理，以确保在时间戳和观测上的对齐。因此，仅凭人手示教，我们就能自动提取相应的SE(3)动作，并通过我们的生成流程将其与高质量生成的机器人示教集成，用于训练机器人策略模型。在实验中，稳健的操作性能不仅证明了生成的机器人示教的质量，也证明了我们数据采集方法的效率和实用性。更多示教可访问：https://rwor.github.io/
英文摘要：
Recent advancements in imitation learning have shown promising results in robotic manipulation, driven by the availability of high-quality training data. To improve data collection efficiency, some approaches focus on developing specialized teleoperation devices for robot control, while others directly use human hand demonstrations to obtain training data. However, the former requires both a robotic system and a skilled operator, limiting scalability, while the latter faces challenges in aligning the visual gap between human hand demonstrations and the deployed robot observations. To address this, we propose a human hand data collection system combined with our hand-to-gripper generative model, which translates human hand demonstrations into robot gripper demonstrations, effectively bridging the observation gap. Specifically, a GoPro fisheye camera is mounted on the human wrist to capture human hand demonstrations. We then train a generative model on a self-collected dataset of paired human hand and UMI gripper demonstrations, which have been processed using a tailored data pre-processing strategy to ensure alignment in both timestamps and observations. Therefore, given only human hand demonstrations, we are able to automatically extract the corresponding SE(3) actions and integrate them with high-quality generated robot demonstrations through our generation pipeline for training robotic policy model. In experiments, the robust manipulation performance demonstrates not only the quality of the generated robot demonstrations but also the efficiency and practicality of our data collection method. More demonstrations can be found at: https://rwor.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
