HRT1:面向移动操作的一次性人机轨迹迁移
HRT1: One-Shot Human-to-Robot Trajectory Transfer for Mobile Manipulation
AI总结:
提出一种四模块人机轨迹迁移系统,使机器人通过一次演示视频学习并迁移移动操作任务,在不同环境中重复执行,实验验证了其有效性。
摘要:
我们提出了一种新颖的人到机器人轨迹迁移系统,使机器人能够通过观看人类演示视频来学习操作物体。该系统由四个模块组成。第一个模块是数据采集模块,旨在使用AR头戴设备从机器人的视角采集人类演示视频。第二个模块是视频理解模块,用于检测物体并从演示视频中提取3D人手轨迹。第三个模块将人手轨迹迁移为3D空间中机器人末端执行器的参考轨迹。最后一个模块利用轨迹优化算法,在机器人配置空间中求解一条能够跟随从人类演示迁移而来的末端执行器轨迹的轨迹。因此,这些模块使机器人能够观看一次人类演示视频,然后在不同环境中重复执行相同的移动操作任务,即使物体放置位置与演示不同。我们在移动操作机器人上进行了不同操作任务的实验,以验证我们系统的有效性。
英文摘要:
We introduce a novel system for human-to-robot trajectory transfer that enables robots to manipulate objects by learning from human demonstration videos. The system consists of four modules. The first module is a data collection module that is designed to collect human demonstration videos from the point of view of a robot using an AR headset. The second module is a video understanding module that detects objects and extracts 3D human-hand trajectories from demonstration videos. The third module transfers a human-hand trajectory into a reference trajectory of a robot end-effector in 3D space. The last module utilizes a trajectory optimization algorithm to solve a trajectory in the robot configuration space that can follow the end-effector trajectory transferred from the human demonstration. Consequently, these modules enable a robot to watch a human demonstration video once and then repeat the same mobile manipulation task in different environments, even when objects are placed differently from the demonstrations. Experiments of different manipulation tasks are conducted on a mobile manipulator to verify the effectiveness of our system
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。