OmniH2O：通用且灵巧的人对人形机器人全身遥操作与学习
OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning
AI总结：
本文提出 OmniH2O 学习系统，以运动学姿态为通用接口实现人形机器人全身遥操作与自主控制，并发布首个全身控制数据集 OmniH2O-6。
摘要：
我们提出 OmniH2O（Omni Human-to-Humanoid），一个基于学习的系统，用于全身人形机器人遥操作与自主控制。OmniH2O 以运动学姿态作为通用控制接口，使人类能够以多种方式控制配备灵巧手的全尺寸人形机器人，包括通过 VR 头显进行实时遥操作、语音指令和 RGB 相机。OmniH2O 还通过从遥操作演示中学习或与 GPT-4 等前沿模型集成来实现完全自主。OmniH2O 在多种真实世界全身任务中通过遥操作或自主方式展示了通用性和灵巧性，例如进行多项体育运动、移动和操作物体以及与人互动。我们开发了一个基于强化学习的 sim-to-real 流程，包括大规模人体运动数据集的重定向与增强、通过模仿特权教师策略来学习可在真实世界部署的稀疏传感器输入策略，以及用于增强鲁棒性和稳定性的奖励设计。我们发布了首个全身人形机器人控制数据集 OmniH2O-6，包含六项日常任务，并展示了从遥操作数据集中学习人形机器人全身技能的能力。
英文摘要：
We present OmniH2O (Omni Human-to-Humanoid), a learning-based system for whole-body humanoid teleoperation and autonomy. Using kinematic pose as a universal control interface, OmniH2O enables various ways for a human to control a full-sized humanoid with dexterous hands, including using real-time teleoperation through VR headset, verbal instruction, and RGB camera. OmniH2O also enables full autonomy by learning from teleoperated demonstrations or integrating with frontier models such as GPT-4. OmniH2O demonstrates versatility and dexterity in various real-world whole-body tasks through teleoperation or autonomy, such as playing multiple sports, moving and manipulating objects, and interacting with humans. We develop an RL-based sim-to-real pipeline, which involves large-scale retargeting and augmentation of human motion datasets, learning a real-world deployable policy with sparse sensor input by imitating a privileged teacher policy, and reward designs to enhance robustness and stability. We release the first humanoid whole-body control dataset, OmniH2O-6, containing six everyday tasks, and demonstrate humanoid whole-body skill learning from teleoperated datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
