RouteMoA: 动态路由无需预推断提升高效混合代理
RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents
AI总结：
RouteMoA通过动态路由机制无需预推断提升混合代理效率，显著降低推理成本和延迟。
摘要：
混合代理（MoA）通过分层协作提升大语言模型（LLM）性能，但其密集拓扑结构带来了成本和延迟问题。现有方法采用LLM裁判过滤响应，但仍然需要所有模型在裁判前进行推理，无法有效降低成本。它们还缺乏模型选择标准，在大规模模型池中表现不佳，其中完整推理成本高昂且可能超出上下文限制。为此，我们提出了RouteMoA，一种高效的混合代理框架，具有动态路由功能。它采用轻量级评分器对查询进行初步筛选，通过预测粗粒度性能来缩小候选集，无需推理即可得到高潜力子集。随后，裁判混合体通过轻量级自我和交叉评估基于现有模型输出，提供后验修正而无需额外推理。最后，模型排名机制通过平衡性能、成本和延迟选择模型。RouteMoA在各种任务和模型池大小上均优于MoA，减少成本89.8%和延迟63.6%在大规模模型池中。
英文摘要：
Mixture-of-Agents (MoA) improves LLM performance through layered collaboration, but its dense topology raises costs and latency. Existing methods employ LLM judges to filter responses, yet still require all models to perform inference before judging, failing to cut costs effectively. They also lack model selection criteria and struggle with large model pools, where full inference is costly and can exceed context limits. To address this, we propose RouteMoA, an efficient mixture-of-agents framework with dynamic routing. It employs a lightweight scorer to perform initial screening by predicting coarse-grained performance from the query, narrowing candidates to a high-potential subset without inference. A mixture of judges then refines these scores through lightweight self- and cross-assessment based on existing model outputs, providing posterior correction without additional inference. Finally, a model ranking mechanism selects models by balancing performance, cost, and latency. RouteMoA outperforms MoA across varying tasks and model pool sizes, reducing cost by 89.8% and latency by 63.6% in the large-scale model pool.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
