arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2611.06817cs.CVcs.RO
收藏

Open-HumanScale:用于扩展具身基础模型的人类与机器人经验的开源方案

Open-HumanWAM: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models

PKU(北京大学);NUS(新加坡国立大学);MIT(麻省理工学院);UCSB(加利福尼亚大学圣巴巴拉分校);NVIDIA(英伟达公司)
Juncheng Ma, Jiaxin Li, Jiankai Tu, Bo Liang, Guodong Lv, Yufan Deng, Junzhe Tang, Liangzhu Yang, Chen Bian, Shukai Gong, Yintianrun Zhang, Zicong Cao, Baohao Z… 展开作者

Juncheng Ma, Jiaxin Li, Jiankai Tu, Bo Liang, Guodong Lv, Yufan Deng, Junzhe Tang, Liangzhu Yang, Chen Bian, Shukai Gong, Yintianrun Zhang, Zicong Cao, Baohao Zhu, Jianxin Bi, Tengfei Yan, Yinyuan Zhang, Kewei Zhang, Xinglong Chen, Kaiqi Chen, Duomin Wang, Xiaotian Tang, Liang Tan, Li Li, Yuan Xu, Haosen Yang, Yuqi Wang, Jingfei Xia, Songyan Huang, Zhiyang Dou, Zhen Dong, Enze Xie, Tat-Seng Chua, Daquan Zhou

AI总结:

本研究提出开源方案Open-HumanScale,构建5万小时以自我为中心的人类视频语料库,开发HumanWAM系列模型,通过纯人类或人机联合预训练提升机器人操作性能,开源相关资源推动具身AI缩放研究。

摘要:

以自我为中心的人类视频为机器人学习提供了可扩展的操作先验来源,但将原始数据规模转化为可泛化的机器人策略需要严格的数据整理和训练方案。我们推出Open-HumanScale,这是首个将以自我为中心的预训练扩展至50000小时的完全开源方案,核心是Open-HumanScale-50K——一个密集标注了手部动作和自然语言的整理后以自我为中心的视频语料库。利用该语料库,我们通过闭环模拟评估系统研究了纯人类预训练,发现了将预训练数据小时数与下游机器人成功率关联的经验对数线性缩放定律。基于这些发现,我们开发了HumanWAM-Vanilla,一个仅在50000小时以自我为中心的人类视频上预训练的世界-动作模型。经机器人后训练后,HumanWAM-Vanilla在RoboTwin 2.0 Clean2Clean任务上达到92.08%的成功率,在Clean2Random任务上达到68.88%,在RoboDojo上的平均成功率为14.17%,平均进度得分为19.00。为进一步拓展能力边界,我们探索了与HumanWAM-Goal的人机联合预训练,这是一个视觉目标条件下的世界-动作模型。混合数据缩放解锁了跨模拟和物理基准的广泛零样本泛化,将后训练的RoboTwin 2.0性能推向新的最先进水平:在Clean2Clean任务上达到92.70%,在Clean2Random任务上大幅跃升至84.60%。在物理机器人上,HumanWAM-Goal进一步展现出涌现的具身上下文学习能力,无需参数更新即可动态调整其操作行为以适应未见的视觉目标。为支持具身AI中可复现的缩放研究,我们开源了Open-HumanScale-50K语料库、数据整理和预训练流程,以及预训练和后训练的HumanWAM检查点、推理代码和模拟评估套件。

英文摘要:

Egocentric human video offers a scalable source of manipulation priors for robot learning, yet translating raw data scale into generalizable robot policies demands rigorous data curation and training recipes. We introduce Open-HumanWAM, the first fully open recipe for scaling egocentric pretraining to 50,000 hours, anchored by Open-HumanWAM-50K---a curated egocentric video corpus densely annotated with hand motion and natural language. Leveraging this corpus, we systematically investigate pure human pretraining via closed-loop simulation evaluations and uncover an empirical log-linear scaling law linking pretraining data hours to downstream robot success rates. Grounded in these findings, we develop HumanWAM-Vanilla, a world--action model pretrained exclusively on 50k hours of egocentric human video. Upon robot post-training, HumanWAM-Vanilla reaches 92.08% success on RoboTwin 2.0 Clean2Clean and 68.88% on Clean2Random, alongside a 14.17% mean success rate and a 19.00 mean progress score on RoboDojo. To push the capability frontier further, we explore joint human--robot pretraining with HumanWAM-Goal, a visual-goal-conditioned world--action model. Hybrid data scaling unlocks broad zero-shot generalization across both simulation and physical benchmarks, propelling post-trained RoboTwin 2.0 performance to new state-of-the-art levels: 92.70% on Clean2Clean and a substantial leap to 84.60% on Clean2Random. On physical robots, HumanWAM-Goal further exhibits emergent embodied in-context learning, dynamically adapting its manipulation behavior to unseen visual goals without parameter updates. To support reproducible scaling research in embodied AI, we open-source the Open-HumanWAM-50K corpus, data curation and pretraining pipelines, alongside the pretrained and post-trained HumanWAM checkpoints, inference code, and simulation evaluation suites.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑