arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.10893cs.LGcs.AI
收藏

交互学习:一种面向真实环境中自适应智能体的数据为中心框架

Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments

Google(谷歌公司);The University of Hong Kong(香港大学)

Hongjin Su, Ruoxi Sun, Jinsung Yoon, Pengcheng Yin, Tao Yu, Sercan Ö. Arık

AI总结:

提出Learn-by-interact数据为中心框架,通过文档合成交互轨迹并反向构建指令,无需人工标注即可自适应LLM智能体,在编码、网页和桌面任务上显著提升性能。

摘要:

由大型语言模型(LLM)驱动的自主智能体有潜力增强人类能力,协助完成从发送电子邮件到执行数据分析等数字任务。现有LLM在此类任务中的能力往往因缺乏其交互环境中相应的高质量智能体数据而受到阻碍。我们提出交互学习(Learn-by-interact),一种数据为中心的框架,旨在无需人工标注的情况下使LLM智能体适应任何给定环境。交互学习基于文档合成智能体-环境交互轨迹,并通过总结或抽象交互历史来构建指令,这一过程称为反向构建。我们通过在基于训练的场景和无训练的情境学习(ICL)中使用合成数据来评估其质量,并为此设计了针对智能体优化的创新检索方法。在SWE-bench、WebArena、OSWorld和Spider2-V上进行的广泛实验,涵盖了真实的编码、网页和桌面环境,展示了交互学习在各种下游智能体任务中的有效性——使用Claude-3.5进行ICL时基线结果提升了高达12.2%,使用Codestral-22B进行训练时提升了19.5%。我们进一步证明了反向构建的关键作用,它为训练提供了高达14.0%的提升。我们的消融研究展示了合成数据在ICL中的效率,以及我们的检索流程相对于传统检索增强生成(RAG)等替代方法的优越性。我们期望随着LLM在真实环境中的日益部署,交互学习将成为智能体数据合成的基础。

英文摘要:

Autonomous agents powered by large language models (LLMs) have the potential to enhance human capabilities, assisting with digital tasks from sending emails to performing data analysis. The abilities of existing LLMs at such tasks are often hindered by the lack of high-quality agent data from the corresponding environments they interact with. We propose Learn-by-interact, a data-centric framework to adapt LLM agents to any given environments without human annotations. Learn-by-interact synthesizes trajectories of agent-environment interactions based on documentations, and constructs instructions by summarizing or abstracting the interaction histories, a process called backward construction. We assess the quality of our synthetic data by using them in both training-based scenarios and training-free in-context learning (ICL), where we craft innovative retrieval approaches optimized for agents. Extensive experiments on SWE-bench, WebArena, OSWorld and Spider2-V spanning across realistic coding, web, and desktop environments show the effectiveness of Learn-by-interact in various downstream agentic tasks -- baseline results are improved by up to 12.2\% for ICL with Claude-3.5 and 19.5\% for training with Codestral-22B. We further demonstrate the critical role of backward construction, which provides up to 14.0\% improvement for training. Our ablation studies demonstrate the efficiency provided by our synthesized data in ICL and the superiority of our retrieval pipeline over alternative approaches like conventional retrieval-augmented generation (RAG). We expect that Learn-by-interact will serve as a foundation for agent data synthesis as LLMs are increasingly deployed at real-world environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑