CoPS:通过可证明的跨任务经验共享赋能LLM智能体
CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing
AI总结:
本文提出CoPS算法,通过可证明的悲观主义策略选择分布匹配的跨任务经验来增强LLM智能体的顺序推理,在多个基准上以更高样本效率超越现有方法。
摘要:
大型语言模型(LLMs)显著推动了智能体系统中的顺序推理,但现有方法仍存在局限。基于反思的推理仅依赖预训练模型中的知识,在全新场景中性能受限;而基于经验辅助的推理通常依赖外部经验,且缺乏选择代表性经验的明确原则。我们通过提出CoPS(Cross-Task Experience Sharing,跨任务经验共享)来解决这些局限,这是一种可泛化的算法,通过跨任务经验共享与选择来增强顺序推理。具体而言,CoPS利用智能体在先前任务上的经验,通过一种可证明的基于悲观主义策略选择分布匹配的经验,以最大化效用,同时最小化分布偏移带来的风险。在Alfworld、Webshop和HotPotQA等基准上的大量实验结果表明,CoPS始终优于最先进的基线方法,并具有优越的样本效率,适用于资源受限的场景。理论上,我们证明该算法的性能既取决于预训练LLM的质量,也取决于智能体任务相关试验分布与LLM生成分布之间的匹配程度。我们的工作弥合了现有顺序推理范式之间的差距,验证了利用跨任务经验的有效性,并为提升智能体在不同任务上的泛化能力和适应性提供了启示。我们的代码可在$\href{https://github.com/uclaml/COPS}{\text{https://github.com/uclaml/COPS}}$获取。
英文摘要:
Sequential reasoning in agent systems has been significantly advanced by large language models (LLMs), yet existing approaches face limitations. Reflection-driven reasoning relies solely on knowledge in pretrained models, limiting performance in novel scenarios, while experience-assisted reasoning often depends on external experiences and lacks clear principles for selecting representative experiences. We address these limitations by proposing CoPS (Cross-Task Experience Sharing), a generalizable algorithm that enhances sequential reasoning by cross-task experience sharing and selection. In detail, CoPS leverages agents' experiences on previous tasks, selecting distribution-matched experiences via a provable pessimism-based strategy to maximize utility while minimizing risks from distribution shifts. Extensive experimental results on benchmarks like Alfworld, Webshop, and HotPotQA demonstrate that CoPS consistently outperforms state-of-the-art baselines, with superior sample efficiency suitable for resource-constrained scenarios. Theoretically, we show that the performance of our algorithm depends on both the quality of the pretrained LLM and the matching between the agent's task-dependent trial distribution and that generated by the LLM. Our work bridges the gap between existing sequential reasoning paradigms and validates the effectiveness of leveraging cross-task experiences, shedding light on the potential to improve agents' generalization and adaptability across diverse tasks. Our codes are available at $\href{https://github.com/uclaml/COPS}{\text{https://github.com/uclaml/COPS}}$.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。