选择后再解决:作为大语言模型代理推理时间优化的范式路由
Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents
AI总结:
研究通过比较六种推理范式在四个前沿LLM和十个基准上的表现,发现推理结构对任务效果有显著影响,提出选择后再解决方法通过轻量级嵌入路由提升任务准确性。
摘要:
当基于LLM的代理在任务上取得进步时,提升来自于模型本身还是围绕其的推理范式?我们通过比较六个推理时间范式(Direct, CoT, ReAct, Plan-Execute, Reflection, ReCode)在四个前沿LLM和十个基准上的表现,共获得约18,000次运行。我们发现推理结构在某些任务上显著提升但在其他任务上损害性能:ReAct在GAIA上比Direct提升44个百分点,而CoT在HumanEval上性能下降15个百分点。没有单一范式主导,oracle per-task选择在平均上比最佳固定范式提升17.1个百分点。受此互补性启发,我们提出选择后再解决方法:在回答每个任务前,轻量级嵌入基于路由选择最合适的范式。在四个模型上,路由将平均准确性从47.6%提升到53.1%,优于最佳固定范式50.3%的2.8个百分点,并恢复多达37%的oracle差距。相比之下,zero-shot self-routing仅在GPT-5上达到67.1%并失败于较弱模型,所有结果均落后于学习路由。我们的结果表明,推理范式选择应由学习的路由进行每任务决策,而非固定架构选择。
英文摘要:
When an LLM-based agent improves on a task, is the gain from the model itself or from the reasoning paradigm wrapped around it? We study this question by comparing six inference-time paradigms, namely Direct, CoT, ReAct, Plan-Execute, Reflection, and ReCode, across four frontier LLMs and ten benchmarks, yielding roughly 18,000 runs. We find that reasoning structure helps dramatically on some tasks but hurts on others: ReAct improves over Direct by 44pp on GAIA, while CoT degrades performance by 15pp on HumanEval. No single paradigm dominates, and oracle per-task selection beats the best fixed paradigm by 17.1pp on average. Motivated by this complementarity, we propose a select-then-solve approach: before answering each task, a lightweight embedding-based router selects the most suitable paradigm. Across four models, the router improves average accuracy from 47.6% to 53.1%, outperforming the best fixed paradigm at 50.3% by 2.8pp and recovering up to 37% of the oracle gap. In contrast, zero-shot self-routing only works for GPT-5 at 67.1% and fails for weaker models, all trailing the learned router. Our results argue that reasoning paradigm selection should be a per-task decision made by a learned router, not a fixed architectural choice.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。