arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2502.03261stat.MLcs.LGcs.NImath.STstat.TH
收藏

CARROT:一种成本感知的速率最优路由器

CARROT: A Cost Aware Rate Optimal Router

Department of Statistics, University of Michigan(密歇根大学统计学系);IBM Research(IBM研究院);MIT-IBM Watson AI Lab(麻省理工-IBM沃森人工智能实验室);Federal University of Minas Gerais(巴西联邦大学矿务分校);Department of Statistics and Actuarial Science, University of Waterloo(滑铁卢大学统计学与精算科学系)

Seamus Somerstep, Felipe Maia Polo, Allysson Flavio Melo de Oliveira, Prattyush Mangal, Mírian Silva, Onkar Bhardwaj, Mikhail Yurochkin, Subha Maity

AI总结:

针对LLM路由中需将查询导向适配且成本最低模型的问题,基于极小极大分析结论提出成本感知速率最优路由器CARROT,同步构建SPROUT数据集,经多基准实验验证其性能优于多种现有路由方法。

摘要:

随着大语言模型(LLM)数量的快速增长,近期学界开始关注LLM路由问题,即把查询导向能给出合适响应的成本最低的LLM。我们对该路由问题开展了极小极大分析,给出了下界,并发现一种能同时预测每个问题的成本与准确率的简单路由器可达到极小极大最优。受此启发,我们提出了CARROT(成本感知速率最优路由器),它基于对各模型成本与性能的估计来选择模型。在提出CARROT的同时,我们还构建了Smart Price-aware ROUTing(SPROUT,智能价格感知路由)数据集,以支持使用最新前沿LLM开展覆盖广泛查询类型的路由研究。我们利用SPROUT以及Routerbench、open-LLM-leaderboard-v2等现有基准,通过实验验证了CARROT相较于多种对比路由器的性能。

英文摘要:

With the rapid growth in the number of Large Language Models (LLMs), there has been a recent interest in LLM routing, or directing queries to the cheapest LLM that can deliver a suitable response. We conduct a minimax analysis of the routing problem, providing a lower bound and finding that a simple router that predicts both cost and accuracy for each question can be minimax optimal. Inspired by this, we introduce CARROT, a Cost AwaRe Rate Optimal rouTer that selects a model based on estimates of the models' cost and performance. Alongside CARROT, we also introduce the Smart Price-aware ROUTing (SPROUT) dataset to facilitate routing on a wide spectrum of queries with the latest state-of-the-art LLMs. Using SPROUT and prior benchmarks such as Routerbench and open-LLM-leaderboard-v2 we empirically validate CARROT's performance against several alternative routers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑