arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2305.16653cs.CLcs.AIcs.LG
收藏

AdaPlanner:基于反馈的自适应规划与语言模型

AdaPlanner: Adaptive Planning from Feedback with Language Models

Georgia Institute of Technology(佐治亚理工学院)

Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, Chao Zhang

AI总结:

AdaPlanner 提出闭环自适应规划方法,让LLM智能体根据环境反馈动态改进计划,并通过代码式提示和技能发现机制减少幻觉与样本需求,在 ALFWorld 和 MiniWoB++ 上以更少样本超越现有方法。

摘要:

大型语言模型(LLMs)近期展示了作为自主智能体执行顺序决策任务的潜力。然而,现有方法大多要么贪婪地采取行动而不进行规划,要么依赖无法适应环境反馈的静态计划。因此,随着问题复杂度和规划时域的增加,LLM智能体的顺序决策性能会下降。我们提出了一种闭环方法 AdaPlanner,使LLM智能体能够根据环境反馈自适应地改进其自生成计划。在 AdaPlanner 中,LLM智能体通过计划内和计划外两种改进策略,根据反馈自适应地改进计划。为了缓解幻觉问题,我们开发了一种代码风格的LLM提示结构,有助于在多种任务、环境和智能体能力下生成计划。此外,我们提出了一种技能发现机制,利用成功计划作为少样本示例,使智能体能够在更少的任务演示下进行规划和改进。我们在 ALFWorld 和 MiniWoB++ 环境中的实验表明,AdaPlanner 分别以少 2 倍和 600 倍的样本量,比最先进的基线方法性能高出 3.73% 和 4.11%。

英文摘要:

Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. However, most existing methods either take actions greedily without planning or rely on static plans that are not adaptable to environmental feedback. Consequently, the sequential decision-making performance of LLM agents degenerates with problem complexity and plan horizons increase. We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback. In AdaPlanner, the LLM agent adaptively refines its plan from feedback with both in-plan and out-of-plan refinement strategies. To mitigate hallucination, we develop a code-style LLM prompt structure that facilitates plan generation across a variety of tasks, environments, and agent capabilities. Furthermore, we propose a skill discovery mechanism that leverages successful plans as few-shot exemplars, enabling the agent to plan and refine with fewer task demonstrations. Our experiments in the ALFWorld and MiniWoB++ environments demonstrate that AdaPlanner outperforms state-of-the-art baselines by 3.73% and 4.11% while utilizing 2x and 600x fewer samples, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑