Autonomous agents powered by language models (LMs) have demonstrated promise in their ability to perform decision-making tasks such as web automation. However, a key limitation remains: LMs, primarily optimized for natural language understanding and generation, struggle with multi-step reasoning, planning, and using environmental feedback when attempting to solve realistic computer tasks. Towards addressing this, we propose an inference-time search algorithm for LM agents to explicitly perform exploration and multi-step planning in interactive web environments. Our approach is a form of best-first tree search that operates within the actual environment space, and is complementary with most existing state-of-the-art agents. It is the first tree search algorithm for LM agents that shows effectiveness on realistic web tasks. On the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%. Our experiments highlight the effectiveness of search for web agents, and we demonstrate that performance scales with increased test-time compute. We conduct a thorough analysis of our results to highlight improvements from search, limitations, and promising directions for future work. Our code and models are publicly released at https://jykoh.com/search-agents.
MARS$^2$: Scaling Multi-Agent Tree Search via Reinforcement Learning for Code Generation
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Shanghai Innovation Institute(上海创新研究院);Harbin Institute of Technology(哈尔滨工业大学);Tsinghua University(清华大学)
AXA AI Research(AXA人工智能研究);Stanford University(斯坦福大学);EPFL, Lausanne, Switzerland(瑞士洛桑联邦理工学院);Polish Academy of Sciences, IBS PAN, Warsaw, Poland(波兰科学院计算机科学研究所)
ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning
University of Hong Kong(香港大学);The Chinese University of Hong Kong(香港中文大学);Jiangxi Science and Technology Normal University(江西科技师范大学);Shenzhen University(深圳大学)
From Prompts to Pavement: LMMs-based Agentic Behavior-Tree Generation Framework for Autonomous Vehicles
Computer Science & Engineering Department, German University in Cairo (GUC), Egypt(德国亚历山大·冯·洪堡大学(开罗分校)计算机科学与工程系,埃及);C-DRiVeS Lab: Cognitive Driving Research in Vehicular Systems, Cairo, Egypt(认知驾驶系统实验室(车辆系统中的认知驾驶研究),开罗,埃及);IAV GmbH, Berlin, Germany(IAV GmbH,柏林,德国)
SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning
DeepWisdom(深度智慧);University of California, Berkeley(加州大学伯克利分校);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of California, San Diego(加州大学圣迭戈分校);South China Normal University(华南师范大学);Stanford University(斯坦福大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Université de Montréal(蒙特利尔大学);Mila(米拉研究所)
Tree-of-Reasoning: Towards Complex Medical Diagnosis via Multi-Agent Reasoning with Evidence Tree
South China University of Technology(华南理工大学);Key Laboratory of Big Data and Intelligent Robot (SCUT), Ministry of Education(大数据与智能机器人重点实验室(SCUT));Hong Kong Polytechnic University(香港理工大学)
Tree of Agents: Improving Long-Context Capabilities of Large Language Models through Multi-Perspective Reasoning
School of Computer and Information Science, Southwest University(西南大学计算机与信息科学学院);School of Information Technology, Murdoch University(默多克大学信息科技学院);School of Computing Technologies, RMIT University(皇家墨尔本理工大学计算技术学院);University of Technology Sydney(悉尼技术大学)
针对大语言模型长上下文处理中的中间信息丢失等问题,提出多智能体推理框架Tree of Agents,通过分块处理、树状协作推理及缓存剪枝策略,在低开销下提升长上下文任务性能,小模型驱动版本即可媲美大型商业模型。
Agents on a Tree: Pathwise Coordination for Multi-Objective Molecular Optimization
College of Information and Science(信息科学学院);Hunan Normal University(湖南师范大学);College of Computer Science(计算机科学学院);Electronic Engineering, Hunan University(电子工程学院,湖南大学)
Monte-Carlo Tree Search for Multi-Agent Pathfinding: Preliminary Results
Moscow Institute of Physics and Technology(莫斯科物理技术学院);AIRI;Federal Research Center "Computer Science and Control" of the Russian Academy of Sciences(俄罗斯科学院计算机科学与控制联邦研究中心)
Tree-Based Formalization of Multi-Agent Complementarity in Human-AI Interactions
Institute of Biomedical Ethics and History of Medicine, University of Zurich(伦理与医学史研究所,苏黎世大学);SUPSI, Dalle Molle Institute for Artificial Intelligence (IDSIA)(SUPSI,达勒莫利人工智能研究所);ETH Zurich(苏黎世联邦理工学院)