可微分的混合智能体激励大型语言模型的群体智能
Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models
AI总结：
提出可微分的混合智能体（DMoA）框架，通过可微分的上下文感知路由机制动态激活智能体，实现推理过程中的弹性协作，并在9个基准上取得最优性能。
摘要：
大型语言模型（LLMs）的最新进展推动了用于复杂推理任务的多智能体系统（MAS）的发展。然而，现有的MAS通常依赖于预定义或预编译的通信拓扑，这限制了它们对动态任务需求的灵活性和适应性。在这项工作中，我们提出了可微分的混合智能体（DMoA），一个自我进化的多智能体框架，能够在推理过程中实现弹性且自适应的智能体协作。不同于静态构建工作流，DMoA在每个推理步骤动态路由和激活智能体，使系统能够隐式模拟多样化的通信拓扑并适应不断变化的需求。为了实现这一点，我们设计了一个可微分的、上下文感知的路由机制，利用循环结构融入历史和上下文信息，以逐步方式产生稀疏的智能体激活。此外，我们引入预测熵作为自监督信号来优化路由过程，实现了无需外部标注的高效测试时自适应。在9个基准上的广泛实验表明，DMoA在实现最先进性能的同时，展现出强大的效率、鲁棒性和集成能力。
英文摘要：
Recent advances in Large Language Models (LLMs) have catalyzed the development of multi-agent systems (MAS) for complex reasoning tasks. However, existing MAS typically rely on pre-defined or pre-compiled communication topologies, which limits their flexibility and adaptability to dynamic task requirements. In this work, we propose Differentiable Mixture-of-Agents (DMoA), a self-evolving multi-agent framework that enables elastic and adaptive agent collaboration during inference. Instead of statically constructing workflows, DMoA dynamically routes and activates agents at each reasoning step, allowing the system to implicitly simulate diverse communication topologies and adapt to evolving demands. To achieve this, we design a differentiable, context-aware routing mechanism that leverages recurrent structures to incorporate historical and contextual information, producing sparse agent activations in a step-wise manner. Furthermore, we introduce predictive entropy as self-supervised signals to optimize the routing process, enabling efficient test-time adaptation without external annotations. Extensive experiments across 9 benchmarks demonstrate that DMoA achieves state-of-the-art performance while exhibiting strong efficiency, robustness, and ensembling capabilities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
