arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.13065cs.ROcs.AIcs.LG
收藏

使用大型语言模型的创造性机器人工具使用

Creative Robot Tool Use with Large Language Models

Carnegie Mellon University(卡内基梅隆大学);Google DeepMind(谷歌DeepMind)

Mengdi Xu, Peide Huang, Wenhao Yu, Shiqi Liu, Xilun Zhang, Yaru Niu, Tingnan Zhang, Fei Xia, Jie Tan, Ding Zhao

AI总结:

本文提出基于LLM的RoboTool系统,通过Analyzer、Planner、Calculator和Coder四个组件,使机器人在隐式物理约束和长期规划任务中实现创造性工具使用,相比传统TAMP方法更灵活高效。

摘要:

工具使用是高级智能的标志,在动物行为和机器人能力中均有体现。本文研究了赋予机器人在涉及隐式物理约束和长期规划的任务中创造性使用工具的能力的可行性。利用大型语言模型(LLMs),我们开发了RoboTool,这是一个接受自然语言指令并输出可执行代码以在模拟和现实环境中控制机器人的系统。RoboTool包含四个关键组件:(i)“Analyzer”,解释自然语言以识别与任务相关的关键概念;(ii)“Planner”,根据语言输入和关键概念生成综合策略;(iii)“Calculator”,计算每个技能的参数;以及(iv)“Coder”,将这些计划转换为可执行的Python代码。我们的结果表明,RoboTool不仅能理解显式或隐式的物理约束和环境因素,还能展示创造性的工具使用。与依赖显式优化的传统任务与运动规划(TAMP)方法不同,我们基于LLM的系统为复杂机器人任务提供了更灵活、高效且用户友好的解决方案。通过大量实验,我们验证了RoboTool在处理那些没有创造性使用工具就无法完成的任务方面表现出色,从而扩展了机器人系统的能力。演示可在我们的项目页面查看:https://creative-robotool.github.io/。

英文摘要:

Tool use is a hallmark of advanced intelligence, exemplified in both animal behavior and robotic capabilities. This paper investigates the feasibility of imbuing robots with the ability to creatively use tools in tasks that involve implicit physical constraints and long-term planning. Leveraging Large Language Models (LLMs), we develop RoboTool, a system that accepts natural language instructions and outputs executable code for controlling robots in both simulated and real-world environments. RoboTool incorporates four pivotal components: (i) an "Analyzer" that interprets natural language to discern key task-related concepts, (ii) a "Planner" that generates comprehensive strategies based on the language input and key concepts, (iii) a "Calculator" that computes parameters for each skill, and (iv) a "Coder" that translates these plans into executable Python code. Our results show that RoboTool can not only comprehend explicit or implicit physical constraints and environmental factors but also demonstrate creative tool use. Unlike traditional Task and Motion Planning (TAMP) methods that rely on explicit optimization, our LLM-based system offers a more flexible, efficient, and user-friendly solution for complex robotics tasks. Through extensive experiments, we validate that RoboTool is proficient in handling tasks that would otherwise be infeasible without the creative use of tools, thereby expanding the capabilities of robotic systems. Demos are available on our project page: https://creative-robotool.github.io/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑