arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.29903cs.RO
收藏

HuMiT:基于紧凑参考跟踪的低延迟全身体人形机器人遥操作

HuMiT: Low-Latency Whole-Body Humanoid Teleoperation via Compact Reference Tracking

Tsinghua IIIS(清华大学智能产业研究院(IIIS));The Chinese University of Hong Kong(香港中文大学);Renmin University of China(中国人民大学);Tsinghua College AI(清华大学AI学院);Tsinghua Weiyang College(清华大学未央书院)

Liu Cao, Botian Xu, Ruoqu Chen, Jiajun Liu, Mingzhi Pei, Zihao Wang, Zhongyu Li, Mengdi Xu

AI总结:

本研究提出HuMiT系统,仅用根状态与四个稀疏关键点位姿的紧凑参考,实现约20ms通信推理延迟,在Unitree G1上验证了其快速反应与协调运动操作能力。

摘要:

实时全身体人形机器人遥操作仍面临挑战,原因在于基于优化的重定向流水线和未来帧条件带来的计算延迟。现有运动跟踪策略通常需要密集参考目标,包括完整关节位置、身体部位变换,甚至未来帧,这使得控制器与昂贵的上游模块紧密耦合。本研究提出HuMiT,一种基于统一跟踪策略的全身体遥操作系统,仅需紧凑参考目标,包括根高度、根速度以及当前帧下四个稀疏关键点的SE(3)位姿,却能达到与依赖更多样参考状态(包括完整关节位置、密集身体部位变换甚至未来帧)的方法相当的跟踪保真度。实验表明,在不同运动类型下,该策略在头戴设备流传输与策略输出间的通信到推理延迟约为20 ms,基于视频的光流延迟中位数为65 ms。我们在Unitree G1人形机器人上验证了该系统,展示了其瞬间反应能力,如接住抛来的球,以及协调的 locomotion-manipulation( locomotion-manipulation 译为“运动-操作”)能力。项目页面:this https URL。

英文摘要:

Real-time whole-body humanoid teleoperation remains challenging due to the computational latency introduced by optimization-based retargeting pipelines and future-frame conditioning. Existing motion tracking policies typically require dense reference targets including full joint positions, body-part transformations, and even future frames, which tightly couples the controller to expensive upstream modules. In this work, we present HuMiT, a whole-body teleoperation system built on a unified tracking policy that requires only a compact reference target, consisting of root height, root velocity, and the SE(3) poses of four sparse keypoints at the current frame, yet achieves competitive tracking fidelity compared to methods relying on more diverse reference states including complete joint positions, dense body-part transformations, and even future frames. Experiments show that our policy achieves a communication-to-inference latency of $\sim$20 ms measured between headset streaming and policy output and a video-based optical flow latency of 65 ms (median) across different motion types. We validate the system on a Unitree G1 humanoid, showcasing split-second reactive responses such as catching a tossed ball, as well as coordinated loco-manipulation capabilities. Project page: https://xiaohu-art.github.io/Track/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑