Large language models (LLMs) are increasingly used in applications where LLM inputs may span many different tasks. Recent work has found that the choice of LLM is consequential, and different LLMs may be good for different input samples. Prior approaches have thus explored how engineers might select an LLM to use for each sample (i.e. routing). While existing routing methods mostly require training auxiliary models on human-annotated data, our work explores whether it is possible to perform unsupervised routing. We propose Smoothie, a weak supervision-inspired routing approach that requires no labeled data. Given a set of outputs from different LLMs, Smoothie constructs a latent variable graphical model over embedding representations of observable LLM outputs and unknown "true" outputs. Using this graphical model, we estimate sample-dependent quality scores for each LLM, and route each sample to the LLM with the highest corresponding score. We find that Smoothie's LLM quality-scores correlate with ground-truth model quality (correctly identifying the optimal model on 9/14 tasks), and that Smoothie outperforms baselines for routing by up to 10 points accuracy.
HiLoRA: Adaptive Hierarchical LoRA Routing for Training-Free Domain Generalization
The Chinese University of Hong Kong, Hong Kong(香港中文大学);FSI Lab, Huawei Technologies Co., Ltd.(华为技术有限公司FSI实验室);Carnegie Mellon University, Pittsburgh, PA, USA(卡内基梅隆大学)
Dynamic Routing as a New Dimension for Test-time Versatility of LLMs
TransformersClub, Faculty of Informatics, Masaryk University(马萨里克大学信息学院 TransformersClub);R&D Centre for Large Language Models, National Institute of Informatics, Japan(日本国立信息学研究所大语言模型研发中心)
MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation
Shanghai Jiao Tong University(上海交通大学);GAIR;Alibaba Group(阿里巴巴集团);University of Science and Technology of China(中国科学技术大学);Shanghai Innovation Institute(上海创新研究院)
DART: Draft-Agreement Routing for Training-Free Adaptive Thinking Budgets in Hybrid Reasoning Models
Korea University(高丽大学);dot(42dot);Konkuk University(建国大学);Yonsei University(延世大学);Soongsil University(崇实大学);Kumoh National Institute of Technology(金乌国立技术学院)
When in Doubt, Consult: Expert Debate for Sexism Detection via Confidence-Based Routing
Department of Computer Science, University of Warwick(沃里克大学计算机科学系);College of Business Studies, Public Authority of Applied Education and Training, Kuwait(卡塔尔应用教育与培训公共机构商学院)