iGPC：面向物体感知人形交互的生成式运动先验

中文摘要 AI 辅助

在非结构化环境中运行的人形机器人必须将鲁棒的全身控制与感知并物理交互周围物体的能力相结合。虽然大规模人体运动数据为自然且多才多艺的人形控制提供了强大的先验，但将这些先验有效转移到感知驱动的物体交互中仍然具有挑战性。为解决这一瓶颈，我们提出了一个框架，将最近提出的生成式预训练控制器（GPC）从一般人体运动扩展到全身人形-环境交互。首先，我们将GPC调整为以场景可供性线索和特权状态信息为条件的交互专家。这些专家利用预训练的人体运动先验，同时学习特定任务的接触行为，包括伸手够取物体、抓取环境支撑以保持稳定，以及推动可移动物体。其次，我们引入了一个感知驱动的学生模型，它保留预训练的GPC策略，并使用机载感官观测从专家那里蒸馏交互技能。为弥合特权专家观测与感官输入之间的差距，我们提出了两个互补的训练目标，使得在蒸馏过程中能够有效适应预训练的运动先验。值得注意的是，我们在多个全身交互任务上的实验表明，大规模生成式人体运动先验为在接触丰富的现实环境中学习可部署的人形交互策略提供了有效基础。

英文摘要

Humanoid robots operating in unstructured environments must combine robust whole-body control with the ability to perceive and physically interact with surrounding objects. While large-scale human motion data provides powerful priors for natural and versatile humanoid control, effectively transferring such priors to perception-driven object interaction remains challenging. To address this bottleneck, we propose a framework that extends the recently proposed Generative Pretrained Controller (GPC) from general human motion to full-body humanoid-environment interaction. First, we adapt GPC into interaction experts conditioned on scene affordance cues and privileged state information. These experts leverage the pretrained human motion prior while learning task-specific contact behaviors, including reaching toward objects, grasping environmental supports for stabilization, and pushing movable objects. Second, we introduce a perception-driven student that retains the pretrained GPC policy and distills interaction skills from the experts using onboard sensory observations. To bridge the gap between privileged expert observations and sensory inputs, we propose two complementary training objectives that enable effective adaptation of the pretrained motion prior during distillation. Notably, our experiments across multiple whole-body interaction tasks demonstrate that large-scale generative human motion priors provide an effective foundation for learning deployable policies for humanoid interactions in contact-rich real-world environments.

发表机构

• Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学）
• Massachusetts Institute of Technology (MIT)（麻省理工学院）
• Minerva Humanoids, Inc.（Minerva Humanoids公司）

机构由 AI 辅助整理，请以论文原文为准。