arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.02805cs.AIcs.CLcs.LG
收藏

图增强的大型语言模型在异步计划推理中的应用

Graph-enhanced Large Language Models in Asynchronous Plan Reasoning

University of Oxford(牛津大学);Alan Turing Institute(阿兰·图灵研究所);Allen Institute for AI(艾伦人工智能研究所);LMU Munich(慕尼黑大学);University of Leeds(利兹大学)

Fangru Lin, Emanuele La Malfa, Valentin Hofmann, Elle Michelle Yang, Anthony Cohn, Janet B. Pierrehumbert

AI总结:

该论文研究了大型语言模型在异步计划推理中的表现,发现现有模型在缺乏说明时表现不佳,提出了一种名为PLaG的图增强技术以提升性能,并揭示了模型在复杂任务中的局限性。

摘要:

规划是人类智能的基本属性。异步计划推理具有挑战性,因为它需要顺序和并行规划来优化时间成本。大型语言模型(LLMs)能成功完成此任务吗?在此,我们提出了首个针对此问题的大规模研究。我们发现,在我们的基准测试AsyncHow中,当未提供关于任务解决过程的说明时,包括GPT-4和LLaMA-2在内的一组代表性的闭源和开源LLMs表现不佳。我们提出了一种名为Plan Like a Graph (PLaG)的新技术,该技术将图与自然语言提示结合,并取得了最先进的结果。我们表明,尽管PLaG可以提升模型性能,但当任务复杂性增加时,LLMs仍会出现急剧的性能下降,突显了利用LLMs模拟数字设备的局限性。我们将我们的研究视为使用LLMs作为高效自主智能体的令人兴奋的一步。我们的代码和数据可在https://github.com/fangru-lin/graph-llm-asynchow-plan获取。

英文摘要:

Planning is a fundamental property of human intelligence. Reasoning about asynchronous plans is challenging since it requires sequential and parallel planning to optimize time costs. Can large language models (LLMs) succeed at this task? Here, we present the first large-scale study investigating this question. We find that a representative set of closed and open-source LLMs, including GPT-4 and LLaMA-2, behave poorly when not supplied with illustrations about the task-solving process in our benchmark AsyncHow. We propose a novel technique called Plan Like a Graph (PLaG) that combines graphs with natural language prompts and achieves state-of-the-art results. We show that although PLaG can boost model performance, LLMs still suffer from drastic degradation when task complexity increases, highlighting the limits of utilizing LLMs for simulating digital devices. We see our study as an exciting step towards using LLMs as efficient autonomous agents. Our code and data are available at https://github.com/fangru-lin/graph-llm-asynchow-plan.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑