arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.15809cs.AIcs.CL
收藏

通过行动学习赋能大语言模型智能体

Empowering Large Language Model Agents through Action Learning

Peking University(北京大学);The University of Hong Kong(香港大学);ByteDance(字节跳动)

Haiteng Zhao, Chang Ma, Guoyin Wang, Jing Su, Lingpeng Kong, Jingjing Xu, Zhi-Hong Deng, Hongxia Yang

AI总结:

本文提出LearnAct框架,通过迭代学习策略让LLM智能体以Python函数形式从失败任务中学习并改进新行动,在Alfworld等环境中显著提升了任务表现。

摘要:

大语言模型(LLM)智能体近年来受到越来越多的关注,但它们在从试错中学习的能力方面仍然有限,而试错学习是智能行为的关键要素。在本工作中,我们认为从经验中学习新行动的能力是推动LLM智能体学习进步的基础。人类能够通过经验学习自然地扩展其行动空间并发展技能,而LLM智能体通常在固定的行动空间内运行,这限制了它们的成长潜力。为应对这些挑战,本研究探索了语言智能体的开放式行动学习。我们提出了一个名为LearnAct的框架,该框架采用迭代学习策略,以Python函数的形式创建和改进行动。在每次迭代中,LLM根据在失败的训练任务中识别出的错误来修订和更新当前可用的行动,从而提高行动的有效性。我们在机器人规划和Alfworld环境中的实验评估表明,在若干训练任务实例上学习之后,我们的开放式行动学习方法显著提升了智能体在同类任务上的表现(例如,在AlfWorld中与ReAct+Reflexion相比提升了32%),这凸显了经验性行动学习在开发更智能的LLM智能体中的重要性。

英文摘要:

Large Language Model (LLM) Agents have recently garnered increasing interest yet they are limited in their ability to learn from trial and error, a key element of intelligent behavior. In this work, we argue that the capacity to learn new actions from experience is fundamental to the advancement of learning in LLM agents. While humans naturally expand their action spaces and develop skills through experiential learning, LLM agents typically operate within fixed action spaces, limiting their potential for growth. To address these challenges, our study explores open-action learning for language agents. We introduce a framework LearnAct with an iterative learning strategy to create and improve actions in the form of Python functions. In each iteration, LLM revises and updates the currently available actions based on the errors identified in unsuccessful training tasks, thereby enhancing action effectiveness. Our experimental evaluations across Robotic Planning and Alfworld environments reveal that after learning on a few training task instances, our approach to open-action learning markedly improves agent performance for the type of task (by 32 percent in AlfWorld compared to ReAct+Reflexion, for instance) highlighting the importance of experiential action learning in the development of more intelligent LLM agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑