As the range of applications for Large Language Models (LLMs) continues to grow, the demand for effective serving solutions becomes increasingly critical. Despite the versatility of LLMs, no single model can optimally address all tasks and applications, particularly when balancing performance with cost. This limitation has led to the development of LLM routing systems, which combine the strengths of various models to overcome the constraints of individual LLMs. Yet, the absence of a standardized benchmark for evaluating the performance of LLM routers hinders progress in this area. To bridge this gap, we present RouterBench, a novel evaluation framework designed to systematically assess the efficacy of LLM routing systems, along with a comprehensive dataset comprising over 405k inference outcomes from representative LLMs to support the development of routing strategies. We further propose a theoretical framework for LLM routing, and deliver a comparative analysis of various routing approaches through RouterBench, highlighting their potentials and limitations within our evaluation framework. This work not only formalizes and advances the development of LLM routing systems but also sets a standard for their assessment, paving the way for more accessible and economically viable LLM deployments. The code and data are available at https://github.com/withmartian/routerbench.
VL-RouterBench: A Benchmark for Vision-Language Model Routing
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University(1 图像处理与模式识别研究所,上海交通大学);The Hong Kong University of Science and Technology (Guangzhou)(2 香港科学与技术大学(广州));The Hong Kong University of Science and Technology(3 香港科学与技术大学)
Department of Statistics, University of Michigan(密歇根大学统计学系);IBM Research(IBM研究院);MIT-IBM Watson AI Lab(麻省理工-IBM沃森人工智能实验室);Federal University of Minas Gerais(巴西联邦大学矿务分校);Department of Statistics and Actuarial Science, University of Waterloo(滑铁卢大学统计学与精算科学系)
TailSieve: Partial-Rollout-Guided Tail Routing for LLM Rollouts
Qwen Business Unit of Alibaba(阿里巴巴通义千问业务部);Carnegie Mellon University(卡内基梅隆大学);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);National University of Singapore(新加坡国立大学);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routing
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);SinapisAI
Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routing
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)
Beyond Gemini-3-Pro: Revisiting LLM Routing and Aggregation at Scale
Shanghai AI Lab(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Fudan University(复旦大学);University of Science and Technology of China(中国科学技术大学);Northwestern Polytechnical University(西北工业大学);Northeastern University(东北大学)
Routing Should Pay for Itself: Sparse Supervision for Economical LLM Routing
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术全国重点实验室);The Hong Kong University of Science and Technology(香港科技大学);SinapisAI
Dynamic Mixed-Precision Routing for Efficient Multi-step LLM Interaction
University of Arizona(亚利桑那大学);University of Pittsburgh(匹兹堡大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of Central Florida(佛罗里达中央大学)
Beyond Top-$k$ Skill Retrieval: Diversity-Aware Skill Routing for LLM Agents
Virginia Tech(弗吉尼亚理工大学);University of Southern California(南加州大学);Adobe Research(Adobe研究院);Dolby Labs(杜比实验室);Texas A&M University(德克萨斯A&M大学);Arizona State University(亚利桑那州立大学)
Department of Electrical and Computer Engineering, University of Pittsburgh(匹兹堡大学电气与计算机工程系);Department of Computer Science, University of Central Florida(中央佛罗里达大学计算机科学系);Department of Computer and Information Science, University of Pennsylvania(宾夕法尼亚大学计算机与信息科学系);Department of Computer Science, University of Maryland(马里兰大学计算机科学系);Department of Computer Science, Florida State University(佛罗里达州立大学计算机科学系)
School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);College of Cyber Science, Nankai University(南开大学网络空间安全学院);The University of Western Australia(西澳大学)