arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.08296cs.AI
收藏

迈向代理系统的科学:扩展性研究

Towards a Science of Scaling Agent Systems

Google Research(谷歌研究院);Massachusetts Institute of Technology(麻省理工学院);Google DeepMind(谷歌DeepMind)
Yubin Kim, Ken Gu, Chanwoo Park, Chunjong Park, Samuel Schmidgall, A. Ali Heydari, Yao Yan, Zhihan Zhang, Yuchen Zhuang, Yun Liu, Mark Malhotra, Paul Pu Liang, … 展开作者

Yubin Kim, Ken Gu, Chanwoo Park, Chunjong Park, Samuel Schmidgall, A. Ali Heydari, Yao Yan, Zhihan Zhang, Yuchen Zhuang, Yun Liu, Mark Malhotra, Paul Pu Liang, Hae Won Park, Yuzhe Yang, Xuhai Xu, Yilun Du, Shwetak Patel, Tim Althoff, Daniel McDuff, Xin Liu

AI总结:

本文通过260种配置评估六种代理基准,探讨代理系统在扩展维度上的性能变化规律,揭示协调、模型能力与任务结构之间的关系,发现协作成功取决于协调与任务结构的匹配程度。

摘要:

代理,一种基于语言模型的系统,能够推理、规划和行动,已被广泛应用于现实任务,但其性能在不同维度扩展时的变化仍不明确。本文引入了代理系统的定量扩展原则作为预测模型,捕捉性能随协调、模型能力和可测量系统和任务因素的变化。在260种配置覆盖六个代理基准、五种经典架构(单代理和四种多代理:独立、集中、去中心化、混合)和三种LLM家族中,进行了受控评估,标准化工具、提示和计算以隔离架构效应。所得模型在所有六个基准上的交叉验证R²为0.373(使用任务基础能力度量时R²=0.413)。我们发现了一种稳健的能力饱和效应和其他模式:(1)协调在单代理基线超过一定性能后会产生边际收益下降;(2)工具密集型任务似乎会产生多代理开销;(3)没有集中验证的架构倾向于传播更多错误。相对于单代理基线的相对性能变化范围从+80.8%在可分解的金融推理到-70.0%在序列规划,表明架构-任务匹配决定了协作成功。该框架在87%的未见配置中识别出最佳性能架构,并在未见前沿模型上显示一致的相对架构偏好。代理的有效性取决于协调与任务结构的匹配程度,不匹配的协调会降低性能。

英文摘要:

Agents, language model-based systems capable of reasoning, planning, and acting are widely adopted in real-world tasks, yet how their performance changes as these systems scale across key dimensions remains underexplored. We introduce quantitative scaling principles for agent systems as a predictive model, capturing how performance varies with coordination, model capability, and measurable system and task factors. Across 260 configurations spanning six agentic benchmarks, five canonical architectures (Single-Agent and four Multi-Agent: Independent, Centralized, Decentralized, Hybrid), and three LLM families, we perform controlled evaluations, standardizing tools, prompts, and compute to isolate architectural effects. The resulting model achieves a cross-validated R^2=0.373 across all six benchmarks (R^2=0.413 with a task-grounded capability metric). We identify a robust capability-saturation effect and additional patterns: (1) a coordination yields diminishing returns once single-agent baselines exceed certain performance; (2) tool-heavy tasks appear to incur multi-agent overhead; and (3) architectures without centralized verification tend to propagate errors more than those with centralized coordination. Relative performance change compared to single-agent baseline ranges from +80.8% on decomposable financial reasoning to -70.0% on sequential planning, demonstrating that architecture-task alignment determines collaborative success. The framework identifies the best-performing architecture for 87% of held-out configurations and shows consistent relative architecture preferences on unseen frontier models. Agent effectiveness depends on alignment between coordination and task structure, and that mismatched coordination degrades the performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑