arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.06646cs.RO
收藏

健忘的大型语言模型:在机器人编程中使用LLMs的经验教训

Forgetful Large Language Models: Lessons Learned from Using LLMs in Robot Programming

Johns Hopkins University(约翰斯·霍普金斯大学)

Juo-Tung Chen, Chien-Ming Huang

AI总结:

本研究通过实证分析LLMs在机器人编程中的执行错误,发现其源于对提示关键信息的健忘,提出提示工程策略并在三个模型上验证有效性,呼吁建立基准测试。

摘要:

大型语言模型为赋能人们编程机器人应用提供了新途径,即通过提示进行代码生成。然而,LLMs生成的代码容易出错。本工作报告了一项初步探索,实证性地刻画了LLMs在机器人编程中产生的常见错误。我们将这些错误分为两个阶段:解释和执行。在本工作中,我们聚焦于执行阶段的错误,并观察到这些错误是由于LLMs对用户提示中提供的关键信息“健忘”所致。基于这一观察,我们提出了旨在减少执行错误的提示工程策略。随后,我们使用三个语言模型——ChatGPT、Bard和LLaMA-2——证明了这些策略的有效性。最后,我们讨论了在机器人编程中使用LLMs的经验教训,并呼吁对由LLM驱动的机器人应用终端用户开发进行基准测试。

英文摘要:

Large language models offer new ways of empowering people to program robot applications-namely, code generation via prompting. However, the code generated by LLMs is susceptible to errors. This work reports a preliminary exploration that empirically characterizes common errors produced by LLMs in robot programming. We categorize these errors into two phases: interpretation and execution. In this work, we focus on errors in execution and observe that they are caused by LLMs being "forgetful" of key information provided in user prompts. Based on this observation, we propose prompt engineering tactics designed to reduce errors in execution. We then demonstrate the effectiveness of these tactics with three language models: ChatGPT, Bard, and LLaMA-2. Finally, we discuss lessons learned from using LLMs in robot programming and call for the benchmarking of LLM-powered end-user development of robot applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑