WB-WAM：用于人形机器人移动操作任务的异构身体-手部预训练

中文摘要 AI 辅助

人形机器人的移动操作任务要求身体和手部行为协调一致，而传统的机器人预训练数据对此类全身动作的覆盖有限。我们提出了WB-WAM，一种世界动作模型，它将显式的全身动作监督纳入生成式视频预训练中。一个共享的物理动作空间整合了来自异构来源的身体、根部和灵巧手部标注，使得能够从1880.2小时的部分标注视频和运动数据中进行联合视频与动作学习。由此产生的先验知识通过PICO中间训练进行细化，并借助辅助的正向运动学监督适配到机器人任务中。我们构建了WB-Datasets以支持这些阶段，其中包含重定向的以自我为中心的人类演示和机器人轨迹，使得任务对齐的人类运动能够补充有限的机器人数据。仿真评估展示了强大的全身任务性能，在HumanoidArena中达到81.9%的成功率，而真实世界实验进一步验证了WB-WAM，在五个任务中平均成功率为84.0%。此外，任务对齐的PICO中间训练提升了下游任务性能，同时减少了对真实机器人演示的需求。这些结果支持异构全身预训练和人类运动迁移作为实现数据高效的人形机器人移动操作任务的实用途径。

英文摘要

Humanoid loco-manipulation demands coordinated body and hand behavior, while conventional robot pre-training data provide limited coverage of such whole-body motion. We present WB-WAM, a World Action Model that incorporates explicit whole-body action supervision into generative video pre-training. A shared physical action space integrates body, root, and dexterous hand annotations from heterogeneous sources, enabling joint video and action learning from 1880.2 hours of partially annotated video and motion data. The resulting priors are refined through PICO mid-training and adapted to robot tasks with auxiliary forward kinematics supervision. We construct WB-Datasets to support these stages with retargeted egocentric human demonstrations and robot trajectories, allowing task-aligned human motion to supplement limited robot data. Evaluations in simulation demonstrate strong whole-body task performance with 81.9% in HumanoidArena, while real-world experiments further validate WB-WAM with 84.0% mean success across five tasks. Moreover, task-aligned PICO mid-training improves downstream task performance while reducing the need for real-robot demonstrations. These results support heterogeneous whole-body pre-training and human motion transfer as a practical route to data-efficient humanoid loco-manipulation.

发表机构

• IIIS, Tsinghua University（清华大学交叉信息研究院）
• The University of Melbourne（墨尔本大学）

机构由 AI 辅助整理，请以论文原文为准。

补充信息

备注

Project Page: https://wb-wam.github.io