arXiv 2506.164752025-07-07cs.ROcs.AIcs.LG

Human2LocoMan：通过人类预训练学习多用途四足机器人操作

Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining

Carnegie Mellon University（卡内基梅隆大学）；Google DeepMind（谷歌DeepMind）；Bosch Center for AI（博世人工智能中心）

Yaru Niu, Yunzhe Zhang, Mingyang Yu, Changyi Lin, Chenhao Li, Yikai Wang, Yuxiang Yang, Wenhao Yu, Tingnan Zhang, Zhenzhen Li, Jonathan Francis, Bingqing Chen, Jie Tan, Ding Zhao

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对四足机器人自主多用途操作技能的可扩展学习挑战，提出跨形态模仿学习系统，通过统一遥操作管道、模块化架构及首个LocoMan操作数据集，结合人类数据预训练，大幅提升真实任务成功率及分布外性能。

摘要：

四足机器人已在复杂环境中展现出令人印象深刻的移动能力，但以可扩展方式为其配备自主多用途操作技能仍是一项重大挑战。在这项工作中，我们引入了一个用于四足机器人操作的跨形态模仿学习系统，利用从人类和LocoMan（一种配备多种操作模式的四足机器人）收集的数据。具体而言，我们开发了一个遥操作和数据收集管道，统一并模块化了人类和机器人的观察与动作空间。为有效利用收集的数据，我们提出了一种高效的模块化架构，支持跨不同形态的结构化模态对齐数据的联合训练和预训练。此外，我们构建了首个针对LocoMan机器人的操作数据集，涵盖单手动和双手动模式下的各种家庭任务，并辅以相应的人类数据集。我们在六项真实世界操作任务中验证了该系统，与基线相比，其总体成功率平均提高41.9%，在分布外（OOD）设置下提高79.7%。使用人类数据进行预训练总体上贡献了38.6%的成功率提升，在OOD设置下提升82.7%，仅用一半的机器人数据即可实现持续更好的性能。我们的代码、硬件和数据已开源：https://human2bots.github.io。

英文摘要：

Quadrupedal robots have demonstrated impressive locomotion capabilities in complex environments, but equipping them with autonomous versatile manipulation skills in a scalable way remains a significant challenge. In this work, we introduce a cross-embodiment imitation learning system for quadrupedal manipulation, leveraging data collected from both humans and LocoMan, a quadruped equipped with multiple manipulation modes. Specifically, we develop a teleoperation and data collection pipeline, which unifies and modularizes the observation and action spaces of the human and the robot. To effectively leverage the collected data, we propose an efficient modularized architecture that supports co-training and pretraining on structured modality-aligned data across different embodiments. Additionally, we construct the first manipulation dataset for the LocoMan robot, covering various household tasks in both unimanual and bimanual modes, supplemented by a corresponding human dataset. We validate our system on six real-world manipulation tasks, where it achieves an average success rate improvement of 41.9% overall and 79.7% under out-of-distribution (OOD) settings compared to the baseline. Pretraining with human data contributes a 38.6% success rate improvement overall and 82.7% under OOD settings, enabling consistently better performance with only half the amount of robot data. Our code, hardware, and data are open-sourced at: https://human2bots.github.io.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。