The proliferation of Large Language Models (LLMs) with varying capabilities and costs has created a need for efficient model selection in AI systems. LLM routers address this need by dynamically choosing the most suitable model for a given query based on task requirements and budget constraints. However, existing routers face challenges in scalability and real-time adaptation, particularly in high-volume online environments. We present Eagle, a novel LLM routing approach that combines global and local ELO ranking modules to overcome these limitations. By evaluating both general and specialized LLM abilities, Eagle provides a scalable, training-free solution that enhances model selection quality while reducing computational overhead. Our experiments across multiple datasets show Eagle consistently outperforms baseline methods, with improvements of up to 23.52 percent in Area Under Curve (AUC) scores. Moreover, Eagle demonstrates remarkable efficiency, requiring only 1/20 of baseline methods' time for initialization and 100 to 200 times faster incremental updates in online scenarios, making it well-suited for dynamic, high-volume online serving environments.
When Routing Reveals Membership: Privacy Leakage from MoE Router Telemetry
Institute of Science Tokyo(东京科学大学);Nanyang Technological University(南洋理工大学);Tohoku University(东北大学);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Shandong University(山东大学)
Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,计算机学院,北京大学);LLM-Core Xiaomi(小米LLM-Core)
Elbow-Based MoE Routing: A Training-Free Inference Time Plugin for Expert Selection
John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·A·保尔森工程与应用科学学院);Department of Mathematics, Harvard University(哈佛大学数学系)
HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts
University of Tennessee at Chattanooga(田纳西大学查塔努加分校);VinUniversity;Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK(格勒诺布尔阿尔卑斯大学、法国国家信息与自动化研究所、法国国家科学研究中心、格勒诺布尔理工学院、LJK实验室);University of Science, Vietnam National University Ho Chi Minh City(胡志明市国家大学科学大学);Salesforce Research Asia(salesforce 亚洲研究院);Singapore Management University(新加坡管理大学)
Meta-Router: Bridging Gold-standard and Preference-based Evaluations in Large Language Model Routing
Department of Statistics, Indiana University Bloomington(印第安纳大学布卢明顿分校统计系);Department of Statistics, North Carolina State University(北卡罗来纳州立大学统计系);Department of Biostatistics, The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心生物统计学系)
Scene-Adaptive Continual Learning for CSI-based Human Activity Recognition with Mixture of Experts
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University(香港理工大学电子与电气工程系);School of Computer Science and Engineering, The Macau University of Science and Technology(澳门科学理工学院计算机科学与工程学院)
Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimization
Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University(符号计算与知识工程重点实验室,MoE,吉林大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
The University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Intel(英特尔);Carnegie Mellon University(卡内基梅隆大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·拉希德人工智能大学);Rutgers University(罗格斯大学)
RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routing
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);SinapisAI
$E^3$-Agent: An Executable and Evolving Agent for Resource Management of Edge Generative Inference
Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai 200240, China(上海交通大学合作中位网创新中心,上海,中国);Department of Broadband Communication, Pengcheng Laboratory, Shenzhen 518000, China(鹏城实验室宽带通信部门,深圳,中国);China Mobile Research Institute, Beijing 100053, China(中国移动研究院,北京,中国)
Robust Mixture-of-Expert Training for Convolutional Neural Networks
Michigan State University(密歇根州立大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);The University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);MIT(麻省理工学院);Harvard University(哈佛大学);UC Santa Barbara(加州大学圣巴巴拉分校);Carnegie Mellon University(卡内基梅隆大学);UIUC(伊利诺伊大学厄巴纳-香槟分校);IBM Research(IBM研究院)
Measured Joules, Learned Routes: Learning to Route for Energy-Efficient LLM Serving
University of Alberta(阿尔伯塔大学);University of California San Diego(加州大学圣地亚哥分校);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));New York University(纽约大学)