arXiv 2504.095322025-10-06cs.ROcs.AI

面向零样本移动操作的、基于多模态基础模型具身动作链推理的人形智能体

Humanoid Agent via Embodied Chain-of-Action Reasoning with Multimodal Foundation Models for Zero-Shot Loco-Manipulation

Embodied AI and Robotics (AIR) Lab, New York University, New York, USA and NYUAD Center for Artificial Intelligence and Robotics, New York University Abu Dhabi, Abu Dhabi, UAE（纽约大学Embodied AI和机器人实验室及纽约大学阿布扎赫尔人工智能与机器人中心）；Harvard AI and Robotics Lab, Harvard University, Boston, USA（哈佛大学人工智能与机器人实验室）；Department of Computer Science, University College London, London, UK（伦敦大学学院计算机科学系）；Department of Computer Science, University of Liverpool, UK（利物浦大学计算机科学系）；NYUAD Center for Artificial Intelligence and Robotics, New York University Abu Dhabi, Abu Dhabi, UAE（纽约大学阿布扎赫尔人工智能与机器人中心）

Congcong Wen, Geeta Chandra Raju Bethala, Yu Hao, Niraj Pudasaini, Hao Huang, Shuaihang Yuan, Baoru Huang, Anh Nguyen, Mengyu Wang, Anthony Tzes, Yi Fang

备注：: website link: https://humanoid-coa.github.io/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对人形机器人移动操作中指令理解与动作生成的难题，提出融合基础模型与具身动作链推理的Humanoid-COA框架，在两款人形机器人的多类任务上实现性能提升与场景泛化。

摘要：

人形机器人移动操作（loco-manipulation）整合全身运动与灵巧操作，仍是机器人领域的一项核心挑战。除全身协调与平衡问题外，核心难点在于理解人类指令并将其转化为连贯的具身动作序列。基础模型的最新进展提供了可迁移的多模态表征与推理能力，但现有研究大多仅局限于单独的运动或操作任务，在人形机器人场景中的适用性有限。\n本文提出Humanoid-COA，这是首个将基础模型推理与具身动作链（Chain-of-Action, CoA）机制相结合、用于零样本移动操作的人形智能体框架。在“感知-推理-行动”范式下，我们的核心贡献在于推理阶段：所提出的CoA机制通过可供性分析、空间推理与全身动作推理，将高层人类指令分解为结构化的运动与操作原语序列。\n我们在开放测试区域与公寓环境中，针对Unitree H1-2和G1两款人形机器人开展了大量实验。结果表明，该框架在操作、运动及移动操作任务上均显著优于现有基线方法，在长时程、非结构化场景中实现了鲁棒的泛化能力。项目主页：https://humanoid-coa.github.io/

英文摘要：

Humanoid loco-manipulation, which integrates whole-body locomotion with dexterous manipulation, remains a fundamental challenge in robotics. Beyond whole-body coordination and balance, a central difficulty lies in understanding human instructions and translating them into coherent sequences of embodied actions. Recent advances in foundation models provide transferable multimodal representations and reasoning capabilities, yet existing efforts remain largely restricted to either locomotion or manipulation in isolation, with limited applicability to humanoid settings. In this paper, we propose Humanoid-COA, the first humanoid agent framework that integrates foundation model reasoning with an Embodied Chain-of-Action (CoA) mechanism for zero-shot loco-manipulation. Within the perception--reasoning--action paradigm, our key contribution lies in the reasoning stage, where the proposed CoA mechanism decomposes high-level human instructions into structured sequences of locomotion and manipulation primitives through affordance analysis, spatial inference, and whole-body action reasoning. Extensive experiments on two humanoid robots, Unitree H1-2 and G1, in both an open test area and an apartment environment, demonstrate that our framework substantially outperforms prior baselines across manipulation, locomotion, and loco-manipulation tasks, achieving robust generalization to long-horizon and unstructured scenarios. Project page: https://humanoid-coa.github.io/

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。