Large Language Model (LLM) routers dynamically select optimal models for given inputs. Existing approaches typically assume access to ground-truth labeled data, which is often unavailable in practice, especially when user request distributions are heterogeneous and unknown. We introduce Routing with Generated Data (RGD), a challenging setting in which routers are trained exclusively on generated queries and answers produced from high-level task descriptions by generator LLMs. We evaluate query-answer routers (using both queries and labels) and query-only routers across four diverse benchmarks and 12 models, finding that query-answer routers degrade faster than query-only routers as generator quality decreases. Our analysis reveals two crucial characteristics of effective generators: they must accurately respond to their own questions, and their questions must produce sufficient performance differentiation among the model pool. We then show how filtering for these characteristics can improve the quality of generated data. We further propose CASCAL, a novel query-only router that estimates model correctness through consensus voting and identifies model-specific skill niches via hierarchical clustering. CASCAL is substantially more robust to generator quality, outperforming the best query-answer router by 4.6% absolute accuracy when trained on weak generator data.
Beyond Top-$k$ Skill Retrieval: Diversity-Aware Skill Routing for LLM Agents
Virginia Tech(弗吉尼亚理工大学);University of Southern California(南加州大学);Adobe Research(Adobe研究院);Dolby Labs(杜比实验室);Texas A&M University(德克萨斯A&M大学);Arizona State University(亚利桑那州立大学)
Beyond Task-Only Matching: Personalized Skill Routing with Counterfactual Evaluation
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);College of Software Engineering, Southeast University(东南大学软件学院);Department of Computer Science and Engineering, The Chinese University of Hong Kong(香港中文大学计算机科学与工程学系);Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系)
Workflow-to-Skill: Skill Creation via Routing-Workflow-Semantics-Attachments Decomposition
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University(航天信息安全部门与可信计算重点实验室,教育部,网络安全科学与工程学院,武汉大学);Nanchang University(南昌大学)
What Keeps Agent Skills from Being Reusable? Evidence from 138K SKILL.md Files
The Graduate Center, City University of New York(纽约城市大学研究生中心);The Ohio State University(俄亥俄州立大学);Hunter College, City University of New York(纽约城市大学亨特学院)
Beyond Prompt or Skill? Attribution-Guided Optimization of Modular LLM Programs
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen)(深圳未来智能网络研究院(深圳FNii));Sun Yat-sen University(中山大学)
HyPAC: Cost-Efficient LLMs-Human Hybrid Annotation with PAC Error Guarantees
Department of Statistics and Data Science, Southern University of Science and Technology, China(统计与数据科学系,南方科技大学);Department of Statistics, University of California at Riverside, USA(统计系,加州大学河滨分校);College of Computing and Data Science, Nanyang Technological University, Singapore(计算与数据科学学院,南洋理工大学);School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China(人工智能学院,香港中文大学(深圳))
Internet and Data Science Lab (IDLab), Ghent University – imec, Ghent, Belgium(互联网与数据科学实验室(IDLab)、根特大学–imec、根特、比利时);Data Science Lab (IDLab), Ghent University – imec, Ghent, Belgium(数据科学实验室(IDLab)、根特大学–imec、根特、比利时)
Demystifying Agent Skills: Why They Work-Until They Don't
Princeton University(普林斯顿大学);UC San Diego(加州大学圣迭戈分校);Stanford University(斯坦福大学);University of Southern California(南加利福尼亚大学);Johns Hopkins University(约翰斯·霍普金斯大学)
Learning What Matters: Prioritized Concept Learning via Relative Error-driven Sample Selection
Mila - Québec AI Institute(魁北克人工智能研究所);University of British Columbia(不列颠哥伦比亚大学);Université de Montréal(蒙特利尔大学);Vector Institute for AI(人工智能矢量研究所);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)