利用预训练模型的语言条件开放词汇移动操作
Language-Conditioned Open-Vocabulary Mobile Manipulation with Pretrained Models
AI总结：
本文提出融合 LLM 与 VLM 的语言条件框架 LOVMM，使机器人依据自然语言指令完成家庭环境中的开放词汇移动操作，并在仿真和桌面任务中展现出优异的零样本泛化与成功率。
摘要：
开放词汇移动操作（OVMM）涉及在不同工作空间中处理新颖且未见过的物体，对于真实世界机器人应用而言仍是一项重大挑战。本文提出一种新颖的语言条件开放词汇移动操作框架，名为 LOVMM，该框架结合大语言模型（LLM）和视觉-语言模型（VLM），以应对家庭环境中的各类移动操作任务。所提方法能够利用自由形式的自然语言指令解决多种 OVMM 任务，例如“把办公室桌上的食品盒扔到角落的垃圾桶里”以及“把床上的瓶子装到客房的箱子里”。在复杂家庭环境中开展的大量仿真实验表明，LOVMM 具有强大的零样本泛化能力和多任务学习能力。此外，该方法还可泛化到多种桌面操作任务，并且与其他最先进方法相比取得了更高的成功率。
英文摘要：
Open-vocabulary mobile manipulation (OVMM) that involves the handling of novel and unseen objects across different workspaces remains a significant challenge for real-world robotic applications. In this paper, we propose a novel Language-conditioned Open-Vocabulary Mobile Manipulation framework, named LOVMM, incorporating the large language model (LLM) and vision-language model (VLM) to tackle various mobile manipulation tasks in household environments. Our approach is capable of solving various OVMM tasks with free-form natural language instructions (e.g. "toss the food boxes on the office room desk to the trash bin in the corner", and "pack the bottles from the bed to the box in the guestroom"). Extensive experiments simulated in complex household environments show strong zero-shot generalization and multi-task learning abilities of LOVMM. Moreover, our approach can also generalize to multiple tabletop manipulation tasks and achieve better success rates compared to other state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
