arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.07587cs.LG
收藏

路由平台:理解并突破LLM路由器的准确性极限

The Routing Plateau: Understanding the Accuracy Limits of LLM Routers

Rice University(莱斯大学);Amazon(亚马逊)

Yifan Lu, Qiyue Zhang, Shenrun Zhang, Zhibo Yu, Zhuang Wang, Hanjie Chen, Jiarong Xing

AI总结:

研究发现多种LLM路由方法存在“路由平台”现象,即准确性趋同且远低于理想路由器,主要原因是可预测性瓶颈;通过增大训练数据、更强编码器和端到端微调可突破平台。

摘要:

LLM路由已成为一种流行的方法,通过为每个查询动态选择模型来改善LLM服务的成本-质量权衡。最近的工作探索了广泛的路由方法,包括基于聚类的路由器、学习分类器、成对排序和基于置信度的方法。我们对五个基准测试中的21种路由方法的广泛研究揭示了一个一致的现象,我们称之为路由平台:许多方法,包括kNN,实现了非常相似的准确性,并收敛到一个狭窄的性能范围,远低于理想路由器。我们的研究表明,平台主要是由可预测性瓶颈引起的:当前路由器主要学习全局平均模型性能趋势,而不是细粒度的查询特定路由信号。因此,它们解决了重叠的简单查询,但共同在需要实例特定路由决策的困难查询上失败。我们进一步研究如何超越平台,发现更大的训练数据集、更强的编码器和端到端微调可以进一步提高路由准确性。这些发现表征了当前路由方法的常见限制,并为社区构建更有效的路由系统提供了见解和可操作的方向。

英文摘要:

LLM routing has become a popular approach to improve the cost-quality trade-off of LLM services by adaptively selecting a model for each query. Recent work has explored a broad range of routing methods, including clustering-based routers, learned classifiers, pairwise ranking, and confidence-based approaches. Our extensive study of 21 routing methods across five benchmarks reveals a consistent phenomenon that we call the routing plateau (Fig. 1): many methods, including kNN, achieve very similar accuracy and converge to a narrow performance range that remains far below the oracle router. Our analysis supports a correctness-prediction bottleneck hypothesis: current routers primarily learn global-average model performance trends rather than fine-grained, query-specific routing signals. As a result, they collectively fail on queries that require instance-specific routing decisions. Moreover, to understand whether the plateau can be alleviated with a better training setup, we construct a 300K-query benchmark (Nine-by-300k). More data, larger encoders, and end-to-end fine-tuning improve eight routers by 1.24 pp on average, but leave the plateau largely intact. These findings suggest that further progress may require inputs beyond the query itself, such as partial output trajectories that reveal how models attempt the task.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑