arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.16079cs.CLcs.AI
收藏

EvolveR:通过经验驱动的生命周期实现自进化大语言模型代理

EvolveR: Self-Evolving LLM Agents through an Experience-Driven Lifecycle

Zhejiang University(浙江大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);East China Normal University(华东师范大学);Fudan University(复旦大学);Central South University(中南大学);Shanghai Innovation Institute(上海创新研究院);Shanghai Jiao Tong University(上海交通大学);University of Science and Technology of China(中国科学技术大学)

Rong Wu, Xiaoman Wang, Jianbiao Mei, Pinlong Cai, Daocheng Fu, Cheng Yang, Licheng Wen, Xuemeng Yang, Yufan Shen, Yuxin Wang, Botian Shi

AI总结:

EvolveR通过闭环经验生命周期实现LLM代理的自进化,结合离线自蒸馏和在线交互,利用策略强化机制迭代优化,提升多跳问答任务性能。

摘要:

当前大语言模型代理在工具使用方面表现优异,但缺乏系统性地从自身经验学习的能力。现有框架主要解决外部知识缺口问题,未能解决更根本的限制:无法迭代优化问题解决策略。本文提出EvolveR框架,通过完整的闭环经验生命周期实现代理自改进。该生命周期包含两个关键阶段:(1) 离线自蒸馏,将代理交互轨迹合成结构化的抽象可重用策略原则;(2) 在线交互,代理与任务互动并主动检索蒸馏的原则以指导决策,积累多样化的行为轨迹。此循环利用策略强化机制迭代更新代理。我们在复杂多跳问答基准测试中展示了EvolveR的有效性,其在强代理基线上的表现更优。我们的工作为能够从外部数据和自身行为后果学习的代理提供了全面蓝图,为更自主和持续改进的系统铺平道路。代码可在https://github.com/Edaizi/EvolveR获取。

英文摘要:

Current Large Language Model (LLM) agents show strong performance in tool use, but lack the crucial capability to systematically learn from their own experiences. While existing frameworks mainly focus on mitigating external knowledge gaps, they fail to address a more fundamental limitation: the inability to iteratively refine problem-solving strategies. In this work, we introduce EvolveR, a framework designed to enable agent to self-improve through a complete, closed-loop experience lifecycle. This lifecycle comprises two key stages: (1) Offline Self-Distillation, where the agent's interaction trajectories are synthesized into a structured repository of abstract, reusable strategic principles; (2) Online Interaction, where the agent interacts with tasks and actively retrieves distilled principles to guide its decision-making, accumulating a diverse set of behavioral trajectories. This loop employs a policy reinforcement mechanism to iteratively update the agent based on its performance. We demonstrate the effectiveness of EvolveR on complex multi-hop question-answering benchmarks, where it achieves superior performance over strong agentic baselines. Our work presents a comprehensive blueprint for agents that learn not only from external data but also from the consequences of their own actions, paving the way for more autonomous and continuously improving systems. Code is available at https://github.com/Edaizi/EvolveR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑