发表机构
Institute of Cyberspace Security, Zhejiang University of Technology; Binjiang Institute of Artificial Intelligence, Zhejiang University of Technology; Hong Kong University of Science and Technology (Guangzhou)（浙江工业大学网络空间安全研究院; 浙江工业大学滨江人工智能研究院; 香港科技大学（广州））
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多候选模型正确回答同一查询导致路由崩溃的问题，提出聚类感知软标签路由框架CASLR，利用掩码softmax和聚类效用分数生成软标签，提升路由准确性与泛化能力，并保持极低延迟。
AI 中文摘要
随着人工智能的快速发展，各种大型语言模型（LLMs）的出现构建了丰富的模型生态系统。然而，这也带来了一个关键挑战：如何为特定的用户查询选择最优模型。LLM路由通过将查询动态分配给候选模型池中最合适的专家来满足这一需求。然而，现有的路由框架通常将此过程简化为标准分类任务；因此，当多个候选模型正确回答同一查询时，会暴露出一个关键漏洞。我们将这种能力重叠形式化为路由噪声，它用任意正确的候选模型误导路由器，最终导致路由崩溃（在未见任务上泛化能力的严重下降）。为解决此问题，我们提出了一种新颖的聚类感知软标签路由（CASLR）框架。CASLR通过用掩码softmax机制替换传统one-hot向量，将评估范式从单个查询的成功转向宏观域共识。具体而言，对于回答错误的专家，我们将其目标概率惩罚为零；对于其余候选者，我们直接基于其全局聚类效用分数计算连续的细粒度软标签。然后，我们使用这些精细的软标签来监督一个轻量级路由器。具体而言，该框架不仅在多个基准测试上展示了优越的准确性，而且在整体平均性能上比Llama-3.3-70B-Instruct高出7.80%。此外，仅1.13秒的极低路由推理延迟进一步证实了CASLR能够以几乎零额外开销实现高效系统调度，同时确保高响应质量。
英文摘要
With the rapid development of artificial intelligence, the emergence of various Large Language Models (LLMs) has created a rich model ecosystem. However, this also brings a key challenge: how to select the optimal model for a specific user query. LLM routing addresses this need by dynamically assigning queries to the most suitable expert in the pool of candidate models. However, existing routing frameworks often simplify this process to a standard classification task; thus, a critical vulnerability is exposed when multiple candidate models correctly answer the same query. We formalize this capability overlap as routing noise, which misleads the router with arbitrarily correct candidate models, ultimately leading to routing collapse (a severe decline in generalization ability on unseen tasks). To address this problem, we propose a novel Cluster-Aware Soft-Labeling Routing (CASLR) framework. CASLR shifts the evaluation paradigm from the success of a single query to macro-domain consensus by replacing traditional one-hot vectors with a masked softmax mechanism. Specifically, for experts who answer incorrectly, we penalize their target probability to zero; for the remaining candidates, we directly compute continuous fine-grained soft labels based on their global clustering utility scores. We then use these refined soft labels to supervise a lightweight router. Specifically, the framework not only demonstrates superior accuracy on multiple benchmarks, but also outperforms Llama-3.3-70B-Instruct by 7.80% in overall average performance. Furthermore, the extremely low routing inference latency of only 1.13s further confirms that CASLR can achieve efficient system scheduling with almost zero additional overhead, while ensuring high response quality.
Comments12 pages, 8 figures
