Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routing
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室)
Large Language Models (LLMs) have demonstrated human-like instruction-following abilities, particularly those exceeding 100 billion parameters. The combined capability of some smaller, resource-friendly LLMs can address most of the instructions that larger LLMs excel at. In this work, we explore how to route the best-performing LLM for each instruction to achieve better overall performance. We develop a new paradigm, constructing capability instructions with model capability representation, user instruction, and performance inquiry prompts to assess the performance. To learn from capability instructions, we introduce a new end-to-end framework called Model Selection with Aptitude Test (Model-SAT), which generates positive and negative samples based on what different models perform well or struggle with. Model-SAT uses a model capability encoder that extends its model representation to a lightweight LLM. Our experiments show that Model-SAT understands the performance dimensions of candidate models and provides the probabilities of their capability to handle various instructions. Additionally, during deployment, a new model can quickly infer its aptitude test results across 50 tasks, each with 20 shots. Model-SAT performs state-of-the-art model routing without candidate inference and in real-world new model-released scenarios. The code is available at https://github.com/Now-Join-Us/CIT-LLM-Routing
Enhancing Instruction-Following Capability of Visual-Language Models by Reducing Image Redundancy
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Kuaishou Technology(快手科技);CAIR, HKISI, Chinese Academy of Sciences(中国科学院香港创新研究院人工智能与机器人创新中心)
Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);University of Science and Technology of China(中国科学技术大学);University of Sydney(悉尼大学);University of Toronto(多伦多大学);Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);Fudan University(复旦大学);Shanghai Innovation Institute(上海创新研究院)
Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimization
Key Laboratory of Symbolic Computation and Knowledge Engineering, MoE, Jilin University(符号计算与知识工程重点实验室,MoE,吉林大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
How Output Format Confounds Data Quality and Capability in Instruction Tuning
University of Chinese Academy of Sciences(中国科学院大学);Pusan National University(釜山国立大学);Shenzhen University of Advanced Technology(深圳先进技术大学);Monash University(莫纳什大学)
SwitchCIT: Switching for Continual Instruction Tuning
Coordinated Science Laboratory(协调科学实验室);Department of Electrical and Computer Engineering(电气与计算机工程系);Department of Mathematics(数学系);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
SEIF: Self-Evolving Reinforcement Learning for Instruction Following
Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University(上海数据科学 key laboratory,计算机科学与人工智能学院,复旦大学);School of Data Science, Fudan University(数据科学学院,复旦大学);Ant Group(蚂蚁集团)
Mitigating LLM Over-Refusal via Dynamic Semantic Routing Calibration
School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);CSIRO(澳大利亚联邦科学与工业研究组织);Center of Excellence for Generative AI, King Abdullah University of Science and Technology(阿卜杜拉国王科技大学生成式人工智能卓越中心)
Beyond Instruction Following: Evaluating Inferential Rule Following of Large Language Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Xiaohongshu Inc AI Lab, AIGility Cloud Innovation(小红书AI实验室);Department of Computer Science and Engineering, Texas A&M University(德克萨斯农工大学计算机科学与工程学院)
Uncovering Intrinsic Capabilities: A Paradigm for Data Curation in Vision-Language Models
Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学(深圳));Hong Kong Baptist University, China(香港 Baptist大学);City University of Hong Kong, China(香港城市大学)
Block-Sparse Attention with Semantic-Geometric Decoupled Routing
Tsinghua University(清华大学);Alibaba Group(阿里巴巴集团);University of Science and Technology of China(中国科学技术大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)