arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.01476cs.AIcs.CLcs.LG
收藏

语言模型代理的树搜索

Tree Search for Language Model Agents

Carnegie Mellon University(卡内基梅隆大学)

Jing Yu Koh, Stephen McAleer, Daniel Fried, Ruslan Salakhutdinov

AI总结:

本文提出一种树搜索算法,用于提升语言模型代理在现实网页任务中的表现,实验显示其在成功率上显著优于基线方法。

摘要:

自主语言模型(LM)代理在执行诸如网页自动化等决策任务方面已展现出潜力。然而,一个关键限制仍然存在:LMs主要优化于自然语言理解和生成,难以进行多步推理、规划以及在解决现实计算机任务时利用环境反馈。为此,我们提出了一种推理时间搜索算法,用于LM代理在交互式网页环境中显式执行探索和多步规划。我们的方法是一种最佳优先树搜索,它在实际环境空间中运行,并与大多数现有最先进的代理互补。这是首个针对LM代理的树搜索算法,能够在现实网页任务中展示有效性。在具有挑战性的VisualWebArena基准上,将我们的搜索算法应用于GPT-4o代理,其成功率相比无搜索的相同基线提高了39.7%,达到了26.4%的最新成功率。在WebArena上,搜索也比基线代理提高了28.0%,达到了19.2%的竞争力成功率。我们的实验突显了搜索在网页代理中的有效性,并展示了性能随测试时间计算量增加而提升。我们对结果进行了深入分析,以突出搜索带来的改进、限制以及未来工作的前景。我们的代码和模型已公开发布在https://jykoh.com/search-agents。

英文摘要:

Autonomous agents powered by language models (LMs) have demonstrated promise in their ability to perform decision-making tasks such as web automation. However, a key limitation remains: LMs, primarily optimized for natural language understanding and generation, struggle with multi-step reasoning, planning, and using environmental feedback when attempting to solve realistic computer tasks. Towards addressing this, we propose an inference-time search algorithm for LM agents to explicitly perform exploration and multi-step planning in interactive web environments. Our approach is a form of best-first tree search that operates within the actual environment space, and is complementary with most existing state-of-the-art agents. It is the first tree search algorithm for LM agents that shows effectiveness on realistic web tasks. On the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%. Our experiments highlight the effectiveness of search for web agents, and we demonstrate that performance scales with increased test-time compute. We conduct a thorough analysis of our results to highlight improvements from search, limitations, and promising directions for future work. Our code and models are publicly released at https://jykoh.com/search-agents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑