arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2311.09830cs.AIcs.CL
收藏

自动化生成基于LLM的PDDL规划中动作选择的提示词

Automating the Generation of Prompts for LLM-based Action Choice in PDDL Planning

Saarland University(萨尔兰大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Aalborg University(奥尔堡大学)

Katharina Stein, Daniel Fišer, Jörg Hoffmann, Alexander Koller

AI总结:

本文提出利用LLM自动从PDDL输入生成自然语言提示词,实现了与手动生成提示词相似的规划性能,并首次对PDDL中的LLM规划进行了广泛评估。

摘要:

大型语言模型(LLMs)彻底改变了多种NLP任务。关于它们能在多大程度上进行推理和规划,目前存在激烈争论。先前的工作在PDDL规划的特定语境下评估了后者,基于手动将三个PDDL领域转换为自然语言(NL)提示词。在此,我们自动化了这一转换步骤,展示了如何利用LLM从PDDL输入自动生成NL提示词。我们自动生成的NL提示词在LLM规划性能上与先前手动生成的相似。除此之外,这种自动化使我们能够运行更大规模的实验,首次对PDDL中的LLM规划性能进行了广泛评估。我们的NL提示词比PDDL提示词和简单的基于模板的NL提示词产生更好的性能。与符号规划器相比,LLM规划远远落后;但在某些领域,我们最佳的LLM配置比使用LM-cut的A*扩展得更远。

英文摘要:

Large language models (LLMs) have revolutionized a large variety of NLP tasks. An active debate is to what extent they can do reasoning and planning. Prior work has assessed the latter in the specific context of PDDL planning, based on manually converting three PDDL domains into natural language (NL) prompts. Here we automate this conversion step, showing how to leverage an LLM to automatically generate NL prompts from PDDL input. Our automatically generated NL prompts result in similar LLM-planning performance as the previous manually generated ones. Beyond this, the automation enables us to run much larger experiments, providing for the first time a broad evaluation of LLM planning performance in PDDL. Our NL prompts yield better performance than PDDL prompts and simple template-based NL prompts. Compared to symbolic planners, LLM planning lags far behind; but in some domains, our best LLM configuration scales up further than A$^\star$ using LM-cut.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑