arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.14652cs.AIcs.CLcs.MA
收藏

MAS-Orchestra:通过整体编排和受控基准理解与改进多智能体推理

MAS-Orchestra: Understanding and Improving Multi-Agent Reasoning Through Holistic Orchestration and Controlled Benchmarks

Salesforce Research(Salesforce研究院);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Massachusetts Institute of Technology(麻省理工学院)

Zixuan Ke, Yifei Ming, Austin Xu, Ryan Chin, Xuan-Phi Nguyen, Prathyusha Jwalapuram, Jiayu Wang, Semih Yavuz, Caiming Xiong, Shafiq Joty

AI总结:

提出MAS-Orchestra框架,将多智能体系统编排形式化为函数调用的强化学习问题,并引入受控基准MASBENCH,揭示MAS收益依赖于任务结构等因素,在数学推理等任务上取得一致改进且效率提升10倍以上。

摘要:

虽然多智能体系统(MAS)通过智能体协调有望提升智能水平,但当前自动MAS设计的方法表现不佳。这些不足源于两个关键因素:(1)方法论复杂性——智能体编排通过顺序的代码级执行进行,限制了全局系统级整体推理,且随智能体复杂性扩展性差;(2)效能不确定性——MAS在未理解相比单智能体系统(SAS)是否有切实益处的情况下被部署。我们提出MAS-Orchestra,一个训练时框架,将MAS编排形式化为具有整体编排的函数调用强化学习问题,一次性生成整个MAS。在MAS-Orchestra中,复杂的、面向目标的子智能体被抽象为可调用函数,从而在隐藏内部执行细节的同时实现系统结构上的全局推理。为了严格研究MAS何时以及为何有益,我们引入了MASBENCH,一个受控基准,沿五个轴表征任务:深度、范围、广度、并行性和鲁棒性。我们的分析揭示,MAS的收益关键取决于任务结构、验证协议以及编排器和子智能体的能力,而非普遍成立。在这些洞察的指导下,MAS-Orchestra在数学推理、多跳问答和基于搜索的问答等公共基准上实现了一致的改进,同时相比强基线实现了超过10倍的效率提升。MAS-Orchestra和MASBENCH共同使得在追求多智能体智能的过程中能够更好地训练和理解MAS。

英文摘要:

While multi-agent systems (MAS) promise elevated intelligence through coordination of agents, current approaches to automatic MAS design under-deliver. Such shortcomings stem from two key factors: (1) methodological complexity - agent orchestration is performed using sequential, code-level execution that limits global system-level holistic reasoning and scales poorly with agent complexity - and (2) efficacy uncertainty - MAS are deployed without understanding if there are tangible benefits compared to single-agent systems (SAS). We propose MASOrchestra, a training-time framework that formulates MAS orchestration as a function-calling reinforcement learning problem with holistic orchestration, generating an entire MAS at once. In MAS-Orchestra, complex, goal-oriented subagents are abstracted as callable functions, enabling global reasoning over system structure while hiding internal execution details. To rigorously study when and why MAS are beneficial, we introduce MASBENCH, a controlled benchmark that characterizes tasks along five axes: Depth, Horizon, Breadth, Parallel, and Robustness. Our analysis reveals that MAS gains depend critically on task structure, verification protocols, and the capabilities of both orchestrator and subagents, rather than holding universally. Guided by these insights, MAS-Orchestra achieves consistent improvements on public benchmarks including mathematical reasoning, multi-hop QA, and search-based QA, while achieving more than 10x efficiency over strong baselines. Together, MAS-Orchestra and MASBENCH enable better training and understanding of MAS in the pursuit of multi-agent intelligence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑