LLMRouter：用于开发、评估和部署LLM路由器的统一基础设施

中文摘要 AI 辅助

没有任何一个大型语言模型（LLM）能在所有查询和预算约束下都达到最优，这使得模型路由对于高性价比部署至关重要。现有路由器采用了多样的形式化方法和实现方式，导致公平比较和扩展变得困难。我们将LLM路由形式化为一个包含五个组件的序列决策过程，这五个组件分别是上下文编码器、模型编码器、评分函数、决策规则和学习信号，涵盖单轮、多轮和个性化路由。基于该形式化方法，我们开发了一条自动流水线，用于构建路由监督信号并在响应质量和推理成本两个维度上联合评估路由器。由此得到的基准xRouteBench涵盖通用LLM、记忆增强、视觉、时间序列和个性化路由任务。我们还推出了LLMRouter，这是一个开源模块化基础设施，包含超过16种代表性路由器。我们的实证研究表明，学习得到的路由器相比最强的固定模型基线有14.6%的相对提升，轻量级路由器在严格成本约束下更具竞争力，而用户条件路由能持续提升个性化效果。

英文摘要

No single large language model (LLM) is optimal across all queries and budget constraints, making model routing essential for cost-effective deployment. Existing routers adopt diverse formulations and implementations, making fair comparison and extension difficult. We present a unified formulation of LLM routing as a sequential decision process characterized by five components: context encoders, model encoders, scoring functions, decision rules, and learning signals, covering single-turn, multi-turn, and personalized routing. Based on this formulation, we develop an automated pipeline for constructing routing supervision and evaluating routers jointly on response quality and inference cost. The resulting benchmark, xRouteBench, spans generic LLM, memory-augmented, vision, time-series, and personalized routing tasks. We further introduce LLMRouter, an open-source modular infrastructure with more than 16 representative routers. Our empirical study shows that learned routers outperform the strongest fixed-model baseline by 14.6% relatively, lightweight routers become more competitive under tight cost constraints, and user-conditioned routing consistently improves personalization.

发表机构

• University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
• University of Maryland, College Park（马里兰大学帕克分校）
• Nanyang Technological University（南洋理工大学）
• Purdue University（普渡大学）
• University of Illinois Chicago（芝加哥伊利诺伊大学）

机构由 AI 辅助整理，请以论文原文为准。