arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.23864cs.AI
收藏

RUMAD:强化统一多智能体辩论

RUMAD: Reinforcement-Unifying Multi-Agent Debate

Tsinghua University(清华大学);Zhejiang University(浙江大学)

Chao Wang, Han Lin, Huaze Tang, Huijing Lin, Wenbo Ding

AI总结:

RUMAD通过强化学习动态控制通信拓扑,提升多智能体辩论的效率和准确性,实现80%以上的token成本降低和跨域泛化能力。

摘要:

多智能体辩论(MAD)系统利用集体智能来增强推理能力,但现有方法难以同时优化准确性、共识形成和计算效率。静态拓扑方法缺乏对任务复杂度变化的适应性,而基于外部LLM的协调可能引入特权知识,影响辩论中立性。本文提出了RUMAD(强化统一多智能体辩论),一种将动态通信拓扑控制建模为强化学习(RL)问题的新框架。RUMAD采用内容无关的观察方案,捕捉高层次辩论动态,避免访问原始智能体推理内容。RUMAD使用多目标奖励来建模解决方案质量、凝聚力和效率。PPO训练的控制器动态调整通信图中的边权重,而双阈值机制可对智能体激活和信息可见性进行精细控制。在MMLU、GSM8K和GPQA基准上的实验评估表明,RUMAD实现了显著的效率提升,将token成本降低超过80%,同时比单个LLM模型和多个MAD基线在推理准确性上有所提升。值得注意的是,仅在MMLU上训练的RUMAD在跨域(OOD)任务中表现出稳健的零样本泛化能力,表明所学的通信策略捕捉了任务无关的有效多智能体协调原理。这些结果确立了RUMAD作为一种高效且稳健的方法,用于在实际资源约束下部署多智能体推理应用。

英文摘要:

Multi-agent debate (MAD) systems leverage collective intelligence to enhance reasoning capabilities, yet existing approaches struggle to simultaneously optimize accuracy, consensus formation, and computational efficiency. Static topology methods lack adaptability to task complexity variations, while external LLM-based coordination risks introducing privileged knowledge that compromises debate neutrality. This work presents RUMAD (Reinforcement-Unifying Multi-Agent Debate), a novel framework that formulates dynamic communication topology control in MAD as a reinforcement learning (RL) problem. RUMAD employs a content-agnostic observation scheme that captures high-level debate dynamics avoiding access to raw agent reasoning content. RUMAD uses a multi-objective reward to model solution quality, cohesion and efficiency. A PPO-trained controller dynamically adjusts edge weights in the communication graph, while a dual-threshold mechanism enables fine-grained control over both agent activation and information visibility. Experimental evaluation across MMLU, GSM8K, and GPQA benchmarks demonstrates that RUMAD achieves substantial efficiency gains, reducing token costs by over 80\%, while still improving reasoning accuracy compared to single LLM model and multiple MAD baselines. Notably, RUMAD trained exclusively on MMLU exhibits robust zero-shot generalization to out-of-domain (OOD) tasks, indicating that the learned communication strategies capture task-independent principles of effective multi-agent coordination. These results establish RUMAD as a efficient and robust approach for deploying multi-agent reasoning application with practical resource constraints.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑