发表机构
AIM3 Lab, Renmin University of China; Qwen Team, Alibaba Inc.; ShanghaiTech University; Beijing Institute for General Artificial Intelligence (BIGAI); Beijing University of Aeronautics and Astronautics（中国人民大学AIM3实验室; 阿里巴巴通义千问团队; 上海科技大学; 北京通用人工智能研究院; 北京航空航天大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
Ego2Robot提出将第一视角人类操控视频转换为机器人训练数据的可扩展流水线，生成18561小时机器人数据，扩展RoboTwin2.0验证其联合预训练可提升机器人分布外泛化能力并经真实部署验证
AI 中文摘要
学习通用机器人操控策略需要大规模多样化的演示数据。第一视角人类操控视频提供了丰富的场景和任务多样性，先前研究表明，将这类视频重定向并渲染为机器人格式数据，在小规模场景下可生成有效的单任务策略。但该方法能否为大规模视觉-语言-动作模型提供预训练益处仍未被探索。我们提出Ego2Robot，这是一种可扩展的流水线，通过动作重定向、机械臂视觉合成和多级质量筛选，将第一视角人类操控视频转换为机器人训练数据。Ego2Robot支持筛选数据集和野外视频，生成了涵盖15种机器人形态的18561小时机器人训练数据，是迄今为止最大的第一视角转机器人数据集。为评估泛化性，我们扩展RoboTwin2.0，加入涵盖视觉外观、场景布局、实体形态和任务语义的解耦扰动轴。实验表明，联合预训练Ego2Robot合成数据与机器人数据，可一致提升多种扰动类型下的分布外泛化能力，其益处已在真实机器人部署中得到验证。项目页面：this https URL
英文摘要
Learning generalizable robot manipulation policies requires large-scale and diverse demonstration data. Egocentric human manipulation videos offer rich scene and task diversity, and prior work has shown that retargeting and rendering such videos into robot-format data can yield effective per-task policies at small scale. However, whether this approach can provide pretraining benefits for vision-language-action models at scale remains unexplored. We present \textbf{Ego2Robot}, a scalable pipeline that converts egocentric human manipulation videos into robot training data through action retargeting, robot-arm visual synthesis, and multi-level quality curation. Ego2Robot supports both curated datasets and in-the-wild videos, producing 18,561 hours of robot training data spanning 15 robot morphologies, making it the largest ego-to-robot dataset to date. To evaluate generalization, we extend RoboTwin2.0 with disentangled perturbation axes covering visual appearance, scene layout, embodiment morphology, and task semantics. Experiments show that joint pretraining on Ego2Robot-synthesized and robot data consistently improves out-of-distribution generalization across multiple perturbation types, with benefits validated on real-robot deployment. Project page: https://www-ye.github.io/ego2robot_blog/
