多智能体计算机使用
Multi-Agent Computer Use
AI总结:
针对单智能体计算机使用代理在复杂长时任务中的不足,提出多智能体计算机使用系统,通过有向无环图分解任务并并行执行,在多个基准测试上提升3.4-25.5%性能,并加速任务完成时间约1.5倍。
摘要:
目前的计算机使用代理(CUA)主要部署为单序列代理。这种设置对于受益于任务分解、并行执行和基于新信息持续重新规划的复杂长时任务来说并不理想。在本文中,我们认为应该转向评估和构建多智能体计算机使用(MACU)系统。这些系统强调规划和并行执行,缓解了单智能体CUA的许多缺点。我们提出了一种通用的多智能体设置,其中管理模型将计算机使用任务分解为有向无环图(DAG),编码子代理的相关依赖关系和目标。在每次迭代中,管理器调度并行的CUA子代理执行DAG就绪前沿上的节点,并根据子代理的新发现持续修订DAG(添加、取消或重写节点)。这种设计将计算机使用的部分可观察环境作为首要挑战:下游代理可能无法重新观察到的信息通过管理器和DAG结构保留并传递。我们证明,MACU在桌面(OSWorld)和网页导航(Online-Mind2Web、WebTailBench、Odysseys)基准测试上始终比强单智能体基线提升3.4-25.5%,表现出更有利的测试时缩放,并解决了单智能体CUA陷入困境的复杂长时任务。在Odysseys(一个长时网页导航基准测试)上,MACU将平均任务完成墙钟时间提高了约1.5倍,证明了其在加速传统缓慢的CUA流程方面的有效性。我们的研究结果强调,多智能体协调是扩展计算机使用代理以更长时间、更有效地工作的一个有前景的方向。我们在https://jykoh.com/multi-agent-computer-use上发布所有代码和交互式可视化。
英文摘要:
Computer use agents (CUAs) today are primarily deployed as single serial agents. This setup is suboptimal for complex long-horizon tasks that benefit from task decomposition, parallel execution, and consistent re-planning based on new information. In this paper, we argue that we should instead move towards evaluating and building multi-agent computer use (MACU) systems. These systems, which emphasize planning and parallel execution, alleviate many of the shortcomings of single-agent CUAs. We propose a general multi-agent setup in which a manager model decomposes computer use tasks as a directed acyclic graph (DAG), encoding relevant dependencies and goals for subagents. At each iteration, the manager dispatches parallel CUA subagents to carry out nodes on the ready frontier of the DAG, and continuously revises the DAG (adding, canceling, or rewriting nodes) as new findings arrive from subagents. This design treats the partially observable environment of computer use as a first class challenge: information that downstream agents may not be able to re-observe are retained and passed forward through the manager and DAG structure. We demonstrate that MACU consistently improves over strong single-agent baselines by $3.4-25.5\%$ on desktop (OSWorld) and web navigation (Online-Mind2Web, WebTailBench, Odysseys) benchmarks, exhibits more favorable test-time scaling, and solves complex long-horizon tasks where single-agent CUAs get stuck. On Odysseys, a long-horizon web navigation benchmark, MACU improves average task completion wall-clock time by ${\sim} 1.5 \times$, demonstrating its efficacy in speeding up traditionally slow CUA pipelines. Our findings highlight that multi-agent coordination is a promising axis for scaling computer use agents to work productively for longer and more effectively. We release all code and interactive visualizations at https://jykoh.com/multi-agent-computer-use.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。