你的 LLM 暗中是互联网的世界模型吗?基于模型的 Web 智能体规划
Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents
AI总结:
针对Web环境中不可逆操作导致树搜索受限的问题,提出基于模型的规划框架WebDreamer,利用LLM作为世界模型模拟候选操作结果,效率提升4-5倍且性能优于基线,并训练了Dreamer-7B模型。
摘要:
基于大型语言模型(LLM)的语言智能体在自动化基于 Web 的任务方面展现了巨大潜力。近期工作表明,结合高级规划算法(如树搜索)对 Web 智能体而言优于反应式规划。然而,与模拟沙盒环境不同,Web 等现实世界环境充斥着不可逆操作。这削弱了回溯的可行性,而回溯是(树)搜索的基石。过度依赖测试时搜索也会损害效率。我们提倡为 Web 智能体采用基于模型的规划,即在执行某操作前,利用世界模型模拟并权衡每个候选操作的结果。我们通过以下方式系统探索这一范式:(1) 提出一个基于模型的规划框架 WebDreamer,该框架利用 LLM 同时充当世界模型和价值函数;(2) 通过可扩展的数据合成流程,训练专门的 LLM 作为世界模型。实证结果表明,WebDreamer 相较于反应式基线取得了显著的性能提升。在沙盒环境(VisualWebArena)中,它与树搜索具有竞争力,且效率提高了 4-5 倍,同时在现实网站(Online-Mind2Web 和 Mind2Web-Live)上也能有效运作。此外,我们训练的世界模型 Dreamer-7B 表现与 GPT-4o 相当,凸显了专门的世界模型在复杂 Web 环境中实现高效且有效规划的潜力。
英文摘要:
Language agents based on large language models (LLMs) have demonstrated great promise in automating web-based tasks. Recent work has shown that incorporating advanced planning algorithms, e.g., tree search, is advantageous over reactive planning for web agents. However, unlike simulated sandbox environments, real-world environments such as the web are rife with irreversible actions. This undermines the feasibility of backtracking, a cornerstone of (tree) search. Overly relying on test-time search also hurts efficiency. We advocate model-based planning for web agents that employs a world model to simulate and deliberate over the outcome of each candidate action before committing to one. We systematically explore this paradigm by (1) Proposing a model-based planning framework, WebDreamer, which employs LLMs to serve as both world models and value functions; (2) Training specialized LLMs as world models with a scalable data synthesis pipeline. Empirical results demonstrate that WebDreamer achieves substantial performance improvements over reactive baselines. It is competitive, while being 4-5 times more efficient, with tree search in sandbox environments (VisualWebArena) and also works effectively on real-world websites (Online-Mind2Web and Mind2Web-Live). Furthermore, our trained world model, Dreamer-7B, performs comparable to GPT-4o, highlighting the potential of specialized world models for efficient and effective planning in complex web environments.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。