arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2406.14655cs.ROcs.AIcs.LG
收藏

HYPERmotion:面向自主移动操作的混合行为规划学习

HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation

Istituto Italiano di Tecnologia(意大利技术研究院);Universita di Genova(热那亚大学)

Jin Wang, Rui Dai, Weijie Wang, Luca Rossini, Francesco Ruscelli, Nikos Tsagarakis

AI总结:

针对人形机器人移动操作任务通用性不足的问题,提出HYPERmotion框架,结合强化学习、全身优化与大语言模型构建层次化规划系统,实现非结构化场景下从文本指令到混合运动的自主执行。

摘要:

让机器人能够在多样化环境中自主执行混合运动,对物料搬运、家务劳动和工作辅助等长周期任务具有重要意义。这需要充分挖掘机器人的内在运动能力,从丰富的环境信息中提取可供性,并规划物理交互行为。尽管近期研究已展现出令人印象深刻的人形机器人全身控制能力,但它们在新任务上仍难以实现通用性与适应性。\n在本研究中,我们提出HYPERmotion框架,该框架可针对不同场景下的任务进行行为学习、选择与规划。我们将强化学习与全身优化相结合,为38个驱动关节生成运动,并构建运动库以存储习得的技能。我们将大语言模型(LLMs)的规划与推理能力应用于复杂移动操作任务,构建了包含一系列基元行为的层次化任务图,以衔接底层执行与高层规划。通过将蒸馏后的空间几何信息与2D观测和视觉语言模型(VLM)的交互,将知识锚定到机器人形态选择器中,从而在单臂或双臂、腿式或轮式运动中选择合适动作。\n仿真与真实环境实验表明,习得的运动能够高效适配新任务,在非结构化场景中展现出从自由文本指令执行任务的高度自主性。相关视频与项目主页:hy-motion.github.io/

英文摘要:

Enabling robots to autonomously perform hybrid motions in diverse environments can be beneficial for long-horizon tasks such as material handling, household chores, and work assistance. This requires extensive exploitation of intrinsic motion capabilities, extraction of affordances from rich environmental information, and planning of physical interaction behaviors. Despite recent progress has demonstrated impressive humanoid whole-body control abilities, they struggle to achieve versatility and adaptability for new tasks. In this work, we propose HYPERmotion, a framework that learns, selects and plans behaviors based on tasks in different scenarios. We combine reinforcement learning with whole-body optimization to generate motion for 38 actuated joints and create a motion library to store the learned skills. We apply the planning and reasoning features of the large language models (LLMs) to complex loco-manipulation tasks, constructing a hierarchical task graph that comprises a series of primitive behaviors to bridge lower-level execution with higher-level planning. By leveraging the interaction of distilled spatial geometry and 2D observation with a visual language model (VLM) to ground knowledge into a robotic morphology selector to choose appropriate actions in single- or dual-arm, legged or wheeled locomotion. Experiments in simulation and real-world show that learned motions can efficiently adapt to new tasks, demonstrating high autonomy from free-text commands in unstructured scenes. Videos and website: hy-motion.github.io/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑