arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.04445cs.NIcs.CLcs.PF
收藏

动态模型路由与级联用于高效LLM推理:综述

Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey

ADAPT Centre(ADAPT中心);School of Computer Science & Statistics(计算机科学与统计学学院);Trinity College Dublin(都柏林三一学院)

Yasmin Moslem, John D. Kelleher

AI总结:

本文综述了多LLM路由与级联方法,分析了不同路由范式及关键权衡,提出框架揭示路由系统在决策时机、信息使用和计算方式上的特性,强调实际系统常整合多种范式以优化性能。

摘要:

大语言模型(LLM)的快速成长带来了推理时智能模型选择的迫切需求。尽管小型模型足以处理常规查询,复杂任务需更强大的模型。然而,静态模型部署无法考虑查询的复杂性和领域,导致性能低下和成本增加。动态路由系统通过适应性选择模型来解决这一挑战。本文系统分析了最先进的多LLM路由与级联方法,不同于混合专家架构,研究了跨多个独立训练的LLM的路由。涵盖了查询难度、人类偏好、聚类、不确定性量化、强化学习、多模态和级联等多种范式。对于每种范式,分析了代表性方法并考察了关键权衡。除了分类,我们引入了一个概念框架,沿三个维度刻画路由系统:决策时机、信息使用和计算方式。此视角表明,实际系统常为组合式,整合多种范式以满足操作约束。分析显示,有效的多LLM路由需平衡竞争性目标。选择最优路由策略取决于部署和计算约束。精心设计的路由系统可通过战略性利用模型的专业能力并最大化效率增益,超越最强大的单个模型。同时,开发能跨多样架构、模态和应用泛化的路由机制仍面临挑战。

英文摘要:

The rapid growth of large language models (LLMs) with diverse capabilities, costs, and domains has created a critical need for intelligent model selection at inference time. While smaller models suffice for routine queries, complex tasks demand more capable models. However, static model deployment does not account for the complexity and domain of incoming queries, leading to suboptimal performance and increased costs. Dynamic routing systems that adaptively select models based on query characteristics have emerged as a solution to this challenge. This survey provides a systematic analysis of multi-LLM routing and cascading approaches, focusing on systems that route queries across a pool of independently trained LLMs at inference time. We cover diverse routing paradigms, including query difficulty, human preferences, clustering, uncertainty quantification, reinforcement learning, multimodality, and cascading. For each paradigm, we analyze representative methods and examine key trade-offs. Beyond taxonomy, we introduce a conceptual framework that characterizes routing systems along three dimensions: when decisions are made, what information is used, and how they are computed. This perspective highlights that practical systems are often compositional, integrating multiple paradigms under operational constraints. Our analysis demonstrates that effective multi-LLM routing requires balancing competing objectives. Choosing the optimal routing strategy depends on deployment and computational constraints. Well-designed routing systems can outperform even the most powerful individual models by strategically leveraging specialized capabilities across models while maximizing efficiency gains. Meanwhile, open challenges remain in developing and evaluating routing mechanisms that generalize across diverse architectures, modalities, and applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑