SeLMRoute：面向大语言模型路由的概率语义证据

中文摘要 AI 辅助

大语言模型（LLM）路由旨在为每个传入查询选择最合适的模型。现有的大多数路由器直接根据查询嵌入、模型表示、偏好数据或相似示例的聚类来学习这一决策。这些方法可能有效，但用于路由的表示很少说明查询实际需要什么。我们提出了SeLMRoute，一种路由框架，它将候选无关的语义证据提取与候选性能学习及部署目标的应用分离开来。决策模型首先评估一组关于查询的可解释问题，例如其推理需求和使用外部知识的情况，每个判断保留为概率分布。由此产生的概率语义状态由一个轻量级监督路由器使用，以估计候选模型的性能。路由目标在性能估计之后应用，这使得相同的语义状态能够支持面向性能和成本意识的决策。在LLMRouterBench（15个数据集，20个候选模型，11,481个查询）上，SeLMRoute实现了$72.08\\% \pm 0.45$的平均准确率，而分组五折袋外评估达到$72.64\\%$，相比之下，最强固定候选为$69.23\\%$。该表示在评估的语义、稠密、词汇和领域级表示中实现了最高的平均性能。在另一个13模型性能-成本设置中，SeLMRoute在所有五个分组分割中均提高了性能，平均PerfGain为$2.66\\%$。我们的代码可从此https URL获取。

英文摘要

Large language model (LLM) routing aims to select the most suitable model for each incoming query. Most existing routers learn this decision directly from query embeddings, model representations, preference data, or clusters of similar examples. Such approaches can be effective, yet the representation used for routing rarely states what a query actually requires. We introduce SeLMRoute, a routing framework that separates the extraction of candidate-independent semantic evidence from the learning of candidate performance and the application of deployment objectives. A decision model first evaluates a set of interpretable questions about the query, such as its reasoning requirements and use of external knowledge, with each judgment retained as a probability distribution. The resulting probabilistic semantic state is used by a lightweight supervised router to estimate candidate model performance. Routing objectives are applied after performance estimation, which allows the same semantic state to support performance-oriented and cost-aware decisions. On the LLMRouterBench (15 datasets, 20 candidate models, 11,481 queries), SeLMRoute achieves an average accuracy of $72.08\% \pm 0.45$, while grouped five-fold out-of-fold evaluation reaches $72.64\%$, compared with $69.23\%$ for the strongest fixed candidate. The representation achieves the highest mean performance among the evaluated semantic, dense, lexical, and domain-level representations. In a separate 13-model performance-cost setting, SeLMRoute improves performance in all five grouped splits, with a mean PerfGain of $2.66\%$. Our code is available at https://github.com/Indigma-Innovations/SeLMRoute.

发表机构

• Indigma Innovations
• Democritus University of Thrace（德谟克利特色雷斯大学）
• Athena Research Center（雅典娜研究中心）

机构由 AI 辅助整理，请以论文原文为准。