arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.06753cs.CL
收藏

选择后再解决:作为大语言模型代理推理时间优化的范式路由

Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents

Heng Zhou, Zelin Tan, Zhemeng Zhang, Yutao Fan, Yibing Lin, Li Kang, Xiufeng Song, Rui Li, Songtao Huang, Ao Yu, Yuchen Fan, Yanxu Chen, Kaixin Xu, Xiaohong Liu… 展开作者

Heng Zhou, Zelin Tan, Zhemeng Zhang, Yutao Fan, Yibing Lin, Li Kang, Xiufeng Song, Rui Li, Songtao Huang, Ao Yu, Yuchen Fan, Yanxu Chen, Kaixin Xu, Xiaohong Liu, Yiran Qin, Philip Torr, Chen Zhang, Zhenfei Yin

AI总结:

研究通过比较六种推理范式在四个前沿LLM和十个基准上的表现,发现推理结构对任务效果有显著影响,提出选择后再解决方法通过轻量级嵌入路由提升任务准确性。

摘要:

当基于LLM的代理在任务上取得进步时,提升来自于模型本身还是围绕其的推理范式?我们通过比较六个推理时间范式(Direct, CoT, ReAct, Plan-Execute, Reflection, ReCode)在四个前沿LLM和十个基准上的表现,共获得约18,000次运行。我们发现推理结构在某些任务上显著提升但在其他任务上损害性能:ReAct在GAIA上比Direct提升44个百分点,而CoT在HumanEval上性能下降15个百分点。没有单一范式主导,oracle per-task选择在平均上比最佳固定范式提升17.1个百分点。受此互补性启发,我们提出选择后再解决方法:在回答每个任务前,轻量级嵌入基于路由选择最合适的范式。在四个模型上,路由将平均准确性从47.6%提升到53.1%,优于最佳固定范式50.3%的2.8个百分点,并恢复多达37%的oracle差距。相比之下,zero-shot self-routing仅在GPT-5上达到67.1%并失败于较弱模型,所有结果均落后于学习路由。我们的结果表明,推理范式选择应由学习的路由进行每任务决策,而非固定架构选择。

英文摘要:

When an LLM-based agent improves on a task, is the gain from the model itself or from the reasoning paradigm wrapped around it? We study this question by comparing six inference-time paradigms, namely Direct, CoT, ReAct, Plan-Execute, Reflection, and ReCode, across four frontier LLMs and ten benchmarks, yielding roughly 18,000 runs. We find that reasoning structure helps dramatically on some tasks but hurts on others: ReAct improves over Direct by 44pp on GAIA, while CoT degrades performance by 15pp on HumanEval. No single paradigm dominates, and oracle per-task selection beats the best fixed paradigm by 17.1pp on average. Motivated by this complementarity, we propose a select-then-solve approach: before answering each task, a lightweight embedding-based router selects the most suitable paradigm. Across four models, the router improves average accuracy from 47.6% to 53.1%, outperforming the best fixed paradigm at 50.3% by 2.8pp and recovering up to 37% of the oracle gap. In contrast, zero-shot self-routing only works for GPT-5 at 67.1% and fails for weaker models, all trailing the learned router. Our results argue that reasoning paradigm selection should be a per-task decision made by a learned router, not a fixed architectural choice.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑