Vision-language models (VLMs) with varying performance and resource requirements are widely deployed, making it difficult for users to select the most appropriate one among numerous VLM candidates. Existing work reveals the performance paradox phenomenon in language models and focuses on routing methods to solve it. However, developing a router for VLM selection is still a critical yet challenging problem, which primarily faces: 1) lack of specialized data, 2) ineffective feature representation, and 3) rigid model space and costly adaptation. In this paper, we construct a multimodal dataset for VLM selection, containing the outputs of seven mainstream VLMs on 32,626 unique image-text queries. We then propose ARMS, a router for VLM selection. ARMS enhances input signals with VLM profiles, employs a simple but effective architecture to improve representations of queries and VLM capabilities. To improve ARMS' adaptation to new VLMs, we propose two extension training strategies: incremental training and independent training. Experimental results on both in-distribution and out-of-distribution test sets demonstrate the effectiveness of ARMS. In particular, using our training strategy, ARMs (only 800M in size) can adapt to a broader VLM space and defeat commercial models like GPT-4o that are hundreds of times larger in scale. Our code, models, and datasets are available in the anonymous repository.
Investigating the Potential of Large Language Model-Based Router Multi-Agent Architectures for Foundation Design Automation: A Task Classification and Expert Selection Study
King Mongkut's University of Technology Thonburi（国王科技大学）
VL-RouterBench: A Benchmark for Vision-Language Model Routing
Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University（1 图像处理与模式识别研究所，上海交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（2 香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（3 香港科学与技术大学）
School of Electronic and Electrical Engineering, Shanghai University of Engineering Science（电子电气工程学院，上海工程技术大学）；Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, Fudan University（计算神经科学与脑启发智能重点实验室，复旦大学）
Model Specific Task Similarity for Vision Language Model Selection via Layer Conductance
School of Computer Science and Technology, University of Science and Technology of China（计算机科学与技术学院，科学技术大学）；School of AI and Data Science, University of Science and Technology of China（人工智能与数据科学学院，科学技术大学）
Agentic Router: An Execution-Grounded Continual Learning Approach With Memory
Zhejiang University（浙江大学）；Zhejiang Lab（之江实验室）；State Grid Hebei Electric Power Co., Ltd.（国网河北省电力有限公司）；Macau University of Science and Technology（澳门科技大学）
Does the Question Really Matter? Training-Free Data Selection for Vision-Language SFT
Nanjing University（南京大学）；Institute of Information Engineering（信息工程研究所）；North University of China（中国北方大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）；LLM-Core Xiaomi（小米LLM-Core）
Adaptive Capacity Allocation for Vision Language Action Fine-tuning
Department of Electrical and Computer Engineering, Seoul National University（电气与计算机工程系，首尔国立大学）；Department of Computer Science and Engineering, Sungkyunkwan University（计算机科学与工程系，全北国立大学）
Confounder-Aware Medical Data Selection for Fine-Tuning Pretrained Vision Models
West China Hospital, Sichuan University（四川大学华西医院）；School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；College of Computer Science, Sichuan University（四川大学计算机学院）；School of Biomedical Engineering, University of Science and Technology of China（中国科学技术大学生物医学工程学院）
Vision Backbone Efficient Selection for Image Classification in Low-Data Regimes
Espace-Dev, IRD, University of Montpellier（Espace-Dev，IRD，蒙彼利埃大学）；Georgia Institute of Technology（佐治亚理工学院）；Field AI；Institute of Computing, Federal University of Rio de Janeiro（里约热内卢联邦大学计算机学院）；Massachusetts Institute of Technology（麻省理工学院）
