arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.02460cs.CLcs.MA
收藏

单代理LLM在多跳推理中优于多代理系统(在相等的思考令牌预算下)

Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets

Stanford University(斯坦福大学)

Dat Tran, Douwe Kiela

AI总结:

在相等的思考令牌预算下,单代理系统在多跳推理任务中表现优于多代理系统,研究通过信息论论证指出单代理系统在信息效率上更具优势,并揭示了多代理系统在计算和上下文利用下降时的竞争性。

摘要:

最近的研究报告了多代理LLM系统(MAS)的强性能,但这些增益常受到测试时计算量增加的干扰。当计算量标准化时,单代理系统(SAS)可以匹配或超越MAS,但这种比较的理论基础和评估方法仍不明确。我们提出一个基于数据处理不等式的信息论论证,表明在固定推理令牌预算和完美上下文利用下,单代理系统更具信息效率。这一观点进一步预测,当单个代理的有效上下文利用下降或更多计算资源被消耗时,多代理系统会更具竞争力。我们在三个模型家族(Qwen3、DeepSeek-R1-Distill-Llama和Gemini 2.5)上进行了受控的实证研究,比较SAS与多个MAS架构在匹配预算下的表现。我们发现,在推理令牌保持不变时,SAS在多跳推理任务中始终能匹配或超越MAS。除了整体性能外,我们还对系统行为和评估方法进行了详细诊断分析。我们识别出API基于预算控制(特别是Gemini 2.5)和标准基准中的显著 artifacts,这些都会放大MAS的显性增益。总体而言,我们的结果表明,对于多跳推理任务,许多报告的多代理系统优势更可能由未计数的计算和上下文效应解释,而非固有架构优势,并突显了理解并明确控制代理系统中计算、上下文和协调之间的权衡的重要性。

英文摘要:

Recent work reports strong performance from multi-agent LLM systems (MAS), but these gains are often confounded by increased test-time computation. When computation is normalized, single-agent systems (SAS) can match or outperform MAS, yet the theoretical basis and evaluation methodology behind this comparison remain unclear. We present an information-theoretic argument, grounded in the Data Processing Inequality, suggesting that under a fixed reasoning-token budget and with perfect context utilization, single-agent systems are more information-efficient. This perspective further predicts that multi-agent systems become competitive when a single agent's effective context utilization is degraded, or when more compute is expended. We test these predictions in a controlled empirical study across three model families (Qwen3, DeepSeek-R1-Distill-Llama, and Gemini 2.5), comparing SAS with multiple MAS architectures under matched budgets. We find that SAS consistently match or outperform MAS on multi-hop reasoning tasks when reasoning tokens are held constant. Beyond aggregate performance, we conduct a detailed diagnostic analysis of system behavior and evaluation methodology. We identify significant artifacts in API-based budget control (particularly in Gemini 2.5) and in standard benchmarks, both of which can inflate apparent gains from MAS. Overall, our results suggest that, for multi-hop reasoning tasks, many reported advantages of multi-agent systems are better explained by unaccounted computation and context effects rather than inherent architectural benefits, and highlight the importance of understanding and explicitly controlling the trade-offs between compute, context, and coordination in agentic systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑