arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.15060cs.ROcs.AI
收藏

CLOT:闭环全局运动跟踪用于全身人形远程操作

CLOT: Closed-Loop Global Motion Tracking for Whole-Body Humanoid Teleoperation

MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University(人工智能联合实验室、AI研究院、上海交通大学);Shanghai AI Laboratory(上海人工智能实验室)

Tengjie Zhu, Guanyu Cai, Yang Zhaohui, Guanzhu Ren, Haohui Xie, ZiRui Wang, Junsong Wu, Jingbo Wang, Xiaokang Yang, Yao Mu, Yichao Yan

AI总结:

CLOT通过高频定位反馈实现闭环全局运动跟踪,解决长时间全身人形远程操作中的姿态漂移问题,结合数据驱动随机化策略和对抗性运动先验,提升远程操作的稳定性和精度。

摘要:

长时间的全身人形远程操作仍然具有挑战性,因为会出现累积的全局姿态漂移,特别是在全尺寸人形上。尽管最近的基于学习的跟踪方法能够实现敏捷和协调的动作,但它们通常在机器人的局部坐标系中操作,并忽视全局姿态反馈,导致在长时间执行过程中出现漂移和不稳定性。在本工作中,我们提出了CLOT,一种实时的全身人形远程操作系统,通过高频定位反馈实现闭环全局运动跟踪。CLOT在闭环中同步操作员和机器人姿态,能够在长时间范围内实现无漂移的人与人形的模仿。然而,直接在强化学习中施加全局跟踪奖励,往往会导致激进且脆弱的修正。为此,我们提出了一种数据驱动的随机化策略,将观察轨迹与奖励评估解耦,从而实现平滑且稳定的全局修正。我们进一步通过对抗性运动先验对策略进行正则化,以抑制不自然的行为。为了支持CLOT,我们收集了20小时精心挑选的人类运动数据用于训练人形远程操作策略。我们设计了一种基于变压器的策略,并将其训练了超过1300个GPU小时。该策略部署在具有31个自由度(不包括手部)的全尺寸人形上。仿真和现实世界实验验证了高动态运动、高精度跟踪和在仿真到现实人形远程操作中的强大鲁棒性。运动数据、演示和代码可在我们的网站上找到。

英文摘要:

Long-horizon whole-body humanoid teleoperation remains challenging due to accumulated global pose drift, particularly on full-sized humanoids. Although recent learning-based tracking methods enable agile and coordinated motions, they typically operate in the robot's local frame and neglect global pose feedback, leading to drift and instability during extended execution. In this work, we present CLOT, a real-time whole-body humanoid teleoperation system that achieves closed-loop global motion tracking via high-frequency localization feedback. CLOT synchronizes operator and robot poses in a closed loop, enabling drift-free human-to-humanoid mimicry over long timehorizons. However, directly imposing global tracking rewards in reinforcement learning, often results in aggressive and brittle corrections. To address this, we propose a data-driven randomization strategy that decouples observation trajectories from reward evaluation, enabling smooth and stable global corrections. We further regularize the policy with an adversarial motion prior to suppress unnatural behaviors. To support CLOT, we collect 20 hours of carefully curated human motion data for training the humanoid teleoperation policy. We design a transformer-based policy and train it for over 1300 GPU hours. The policy is deployed on a full-sized humanoid with 31 DoF (excluding hands). Both simulation and real-world experiments verify high-dynamic motion, high-precision tracking, and strong robustness in sim-to-real humanoid teleoperation. Motion data, demos and code can be found in our website.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑