EgoMI: 从第一人称人类演示中学习主动视觉和全身操作
EgoMI: Learning Active Vision and Whole-Body Manipulation from Egocentric Human Demonstrations
AI总结：
EgoMI通过同步捕捉头部和末端执行器轨迹，实现从第一人称人类演示中学习主动视觉与全身操作，有效弥合人机身体差距，提升半人形机器人模仿学习性能。
摘要：
从人类演示中学习模仿提供了机器人技能获取的有希望的方法，但第一人称人类数据引入了根本性的挑战，因为存在身体差距。在操作过程中，人类主动协调头部和手部运动，持续重新定位视角，并使用预动作视觉固定搜索策略来定位相关物体。这些行为创造了动态、任务驱动的头部运动，静态机器人感知系统无法复制，导致显著的分布偏移，从而降低策略性能。我们提出了EgoMI（第一人称操作接口），一个框架，能够捕捉操作任务中的同步末端执行器和主动头部轨迹，产生可以重新定位到兼容半人形机器人 embodiment 的数据。为处理快速且广范围的头部视角变化，我们引入了记忆增强的策略，能够选择性地整合历史观察。我们在配备可动摄像头头部的双臂机器人上评估了我们的方法，并发现具有显式头部运动建模的策略始终优于基线方法。结果表明，使用EgoMI的协调手眼学习有效弥合了人机身体差距，实现了对半人形 embodiment 上稳健模仿学习的提升。项目页面：https://egocentric-manipulation-interface.github.io
英文摘要：
Imitation learning from human demonstrations offers a promising approach for robot skill acquisition, but egocentric human data introduces fundamental challenges due to the embodiment gap. During manipulation, humans actively coordinate head and hand movements, continuously reposition their viewpoint and use pre-action visual fixation search strategies to locate relevant objects. These behaviors create dynamic, task-driven head motions that static robot sensing systems cannot replicate, leading to a significant distribution shift that degrades policy performance. We present EgoMI (Egocentric Manipulation Interface), a framework that captures synchronized end-effector and active head trajectories during manipulation tasks, resulting in data that can be retargeted to compatible semi-humanoid robot embodiments. To handle rapid and wide-spanning head viewpoint changes, we introduce a memory-augmented policy that selectively incorporates historical observations. We evaluate our approach on a bimanual robot equipped with an actuated camera head and find that policies with explicit head-motion modeling consistently outperform baseline methods. Results suggest that coordinated hand-eye learning with EgoMI effectively bridges the human-robot embodiment gap for robust imitation learning on semi-humanoid embodiments. Project page: https://egocentric-manipulation-interface.github.io
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
