arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.11604cs.ROcs.AIcs.CLcs.HCcs.LG
收藏

语言模型作为零样本轨迹生成器

Language Models as Zero-Shot Trajectory Generators

Imperial College London(伦敦帝国学院)

Teyun Kwon, Norman Di Palo, Edward Johns

AI总结:

本文研究GPT-4能否零样本直接生成机器人低层操作轨迹,提出任务无关提示并在30项真实任务上验证其具备低层控制、失败检测与重规划能力。

摘要:

大型语言模型(LLM)近期在能够使用一组低层技能时,已展现出作为机器人高层规划器的前景。然而,人们通常认为LLM不具备足够知识,无法用于生成低层轨迹本身。在本研究中,我们全面审视这一假设,并考察LLM(GPT-4)在仅能使用目标检测与分割视觉模型的情况下,能否直接预测操作任务的密集末端执行器位姿序列。我们设计了一个单一、与任务无关的提示,其中没有任何上下文示例、运动基元或外部轨迹优化器。随后,我们研究了该方法在30项基于语言的真实世界任务上的表现,例如“打开瓶盖”和“用海绵擦拭盘子”,并考察该提示中哪些设计选择最重要。我们的结论提高了LLM在机器人学中被假定的能力上限;我们首次揭示,LLM确实具备足以应对一系列常见任务的低层机器人控制理解能力,并且还能够检测失败,随后据此重新规划轨迹。视频、提示和代码可在 https://www.robot-learning.uk/language-models-trajectory-generators 获取。

英文摘要:

Large Language Models (LLMs) have recently shown promise as high-level planners for robots when given access to a selection of low-level skills. However, it is often assumed that LLMs do not possess sufficient knowledge to be used for the low-level trajectories themselves. In this work, we address this assumption thoroughly, and investigate if an LLM (GPT-4) can directly predict a dense sequence of end-effector poses for manipulation tasks, when given access to only object detection and segmentation vision models. We designed a single, task-agnostic prompt, without any in-context examples, motion primitives, or external trajectory optimisers. Then we studied how well it can perform across 30 real-world language-based tasks, such as "open the bottle cap" and "wipe the plate with the sponge", and we investigated which design choices in this prompt are the most important. Our conclusions raise the assumed limit of LLMs for robotics, and we reveal for the first time that LLMs do indeed possess an understanding of low-level robot control sufficient for a range of common tasks, and that they can additionally detect failures and then re-plan trajectories accordingly. Videos, prompts, and code are available at: https://www.robot-learning.uk/language-models-trajectory-generators.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑