arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.09435cs.RO

面向人类指令与反馈的序列操作规划:基于大语言模型的行为树生成

Behavior Tree Generation using Large Language Models for Sequential Manipulation Planning with Human Instructions and Feedback

Technical University of Munich(慕尼黑工业大学)

Jicong Ao, Yansong Wu, Fan Wu, Sami Haddadin

AI总结:

本研究提出首个由人类引导的基于大语言模型的行为树生成框架,支持人类指令启动与反馈优化,生成的行为树可在真实机器人装配平台执行,兼顾工具使用细粒度知识,在多维度指标上验证了有效性。

摘要:

本研究提出一种基于大语言模型(LLM)的行为树(BT)生成框架,融合二者优势用于序列操作规划。为实现人机协同任务规划、降低非专业人员的机器人编程门槛,该框架接收人类指令以启动动作序列生成,并通过人类反馈在运行时优化BT生成过程。框架内所有方法均在真实机器人装配案例中测试,采用西门子机器人装配挑战赛(Siemens Robot Assembly Challenge)的齿轮组模型,使用配备换刀机构的单机械臂(柔性制造中的常见配置),以实现对多种物体的稳定抓取。实验从成功率、逻辑一致性、可执行性、耗时与token消耗量维度进行评估。据我们所知,这是首个由人类引导的基于LLM的BT生成框架,它整合了多种利用LLM完整生成BT的可行方式,生成的BT可在真实测试平台上执行,且纳入了工具使用的细粒度知识。

英文摘要:

In this work, we propose an LLM-based BT generation framework to leverage the strengths of both for sequential manipulation planning. To enable human-robot collaborative task planning and enhance intuitive robot programming by nonexperts, the framework takes human instructions to initiate the generation of action sequences and human feedback to refine BT generation in runtime. All presented methods within the framework are tested on a real robotic assembly example, which uses a gear set model from the Siemens Robot Assembly Challenge. We use a single manipulator with a tool-changing mechanism, a common practice in flexible manufacturing, to facilitate robust grasping of a large variety of objects. Experimental results are evaluated regarding success rate, logical coherence, executability, time consumption, and token consumption. To our knowledge, this is the first human-guided LLM-based BT generation framework that unifies various plausible ways of using LLMs to fully generate BTs that are executable on the real testbed and take into account granular knowledge of tool use.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑