arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.00344cs.CLstat.AP
收藏

Agent Q-Mix: 通过强化学习选择LLM多智能体系统的合适动作

Agent Q-Mix: Selecting the Right Action for LLM Multi-Agent Systems through Reinforcement Learning

University of California Los Angeles(加州大学洛杉矶分校);University of Washington(华盛顿大学);Northwestern University(西北大学)

Eric Hanchen Jiang, Levina Li, Rui Sun, Xiao Liang, Yubei Li, Yuchen Wu, Haozheng Luo, Hengli Li, Zhi Zhang, Zhaolu Kang, Kai-Wei Chang, Ying Nian Wu

AI总结:

本文提出Agent Q-Mix框架,通过强化学习解决多智能体系统中拓扑选择问题,实现去中心化通信决策,提升任务准确率与token效率。

摘要:

大型语言模型(LLMs)在完成各种任务上表现出色,但解决复杂问题通常需要多个智能体的协调,提出了如何有效选择和连接这些智能体的问题。本文提出Agent Q-Mix,一种强化学习框架,将拓扑选择重新表述为合作多智能体强化学习(MARL)问题。方法通过QMIX值因子化学习去中心化通信决策,每个智能体从一组通信动作中选择,共同诱导轮次通信图。核心是结合拓扑感知GNN编码器、GRU记忆和每个智能体的Q头,在集中训练与去中心化执行(CTDE)范式下。框架优化一个平衡任务准确率与token成本的奖励函数。在编码、推理和数学七个核心基准上,Agent Q-Mix在任务准确率上最高,同时展示出更高效的token使用和更强的抗失败鲁棒性。特别地,在挑战性的人类最后考试(HLE)中,使用Gemini-3.1-Flash-Lite作为基础模型时,Agent Q-Mix达到20.8%的准确率,优于Microsoft Agent Framework(19.2%)和LangGraph(19.2%),随后是AutoGen和Lobster。这些结果证明了学习的去中心化拓扑优化在多智能体推理方面的有效性。

英文摘要:

Large Language Models (LLMs) have shown remarkable performance in completing various tasks. However, solving complex problems often requires the coordination of multiple agents, raising a fundamental question: how to effectively select and interconnect these agents. In this paper, we propose \textbf{Agent Q-Mix}, a reinforcement learning framework that reformulates topology selection as a cooperative Multi-Agent Reinforcement Learning (MARL) problem. Our method learns decentralized communication decisions using QMIX value factorization, where each agent selects from a set of communication actions that jointly induce a round-wise communication graph. At its core, Agent Q-Mix combines a topology-aware GNN encoder, GRU memory, and per-agent Q-heads under a Centralized Training with Decentralized Execution (CTDE) paradigm. The framework optimizes a reward function that balances task accuracy with token cost. Across seven core benchmarks in coding, reasoning, and mathematics, Agent Q-Mix achieves the highest average accuracy compared to existing methods while demonstrating superior token efficiency and robustness against agent failure. Notably, on the challenging Humanity's Last Exam (HLE) using Gemini-3.1-Flash-Lite as a backbone, Agent Q-Mix achieves 20.8\% accuracy, outperforming Microsoft Agent Framework (19.2\%) and LangGraph (19.2\%), followed by AutoGen and Lobster by OpenClaw. These results underscore the effectiveness of learned, decentralized topology optimization in pushing the boundaries of multi-agent reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑