Foundation language model deployments often include auxiliary guard-rail models to filter or classify text, detecting jailbreak attempts, biased or toxic output, or ensuring topic adherence. These additional models increase the complexity and cost of model inference, especially since many are also large language models. To address this issue, we explore training-free model merging techniques to consolidate these models into a single, multi-functional model. We propose Heterogeneous Multi-Class Model Merging (HM3) as a simple technique for merging multi-class classifiers with heterogeneous label spaces. Unlike parameter-efficient fine-tuning techniques like LoRA, which require extensive training and add complexity during inference, recent advancements allow models to be merged in a training-free manner. We report promising results for merging BERT-based guard models, some of which attain an average F1-score higher than the source models while reducing the inference time by up to 44%. We introduce self-merging to assess the impact of reduced task-vector density, finding that the more poorly performing hate speech classifier benefits from self-merging while higher-performing classifiers do not, which raises questions about using task vector reduction for model tuning.
Rethinking Heterogeneous LLM Merging: A Weighted Model Averaging Perspective
University of Science and Technology of China(中国科学技术大学);School of Computer Science and Technology, University of Science and Technology of China(中国科学技术大学计算机科学与技术学院)
PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所);Pengcheng Laboratory(鹏城实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Data Science and Artificial Intelligence Research Institute, China United Network Communications Group Co., Ltd.(中国联合网络通信集团有限公司数据科学与人工智能研究院)
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
School of Computer Science & Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL)(华中科技大学与平安产险联合实验室);Ping An Property & Casualty Insurance Company of China, Ltd.(中国平安财产保险股份有限公司);The Chinese University of Hong Kong(香港中文大学)
Model Assembly Learning with Heterogeneous Layer Weight Merging
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);Yingcai Honors College, University of Electronic Science and Technology of China(电子科技大学英才学院)
AdaMMS: Model Merging for Heterogeneous Multimodal Large Language Models with Unsupervised Coefficient Optimization
Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University(计算机科学与技术系,人工智能研究院,清华大学);Institute for AI Industry Research (AIR), Tsinghua University(人工智能产业研究院(AIR),清华大学);State Key Laboratory of Multimedia Information Processing, Peking University(多媒体信息处理国家重点实验室,北京大学);School of Software Microelectronics, Peking University(软件微电子学院,北京大学);Institute of Intelligent Computing, Alibaba Group(智能计算研究院,阿里巴巴集团);Shanghai Artificial Intelligence Laboratory, Shanghai, China(上海人工智能实验室,上海,中国);Jiangsu Collaborative Innovation Center for Language Competence, Jiangsu, China(江苏省语言能力协同创新中心,江苏,中国);ModelTC Open Source Organization, Beijing, China(ModelTC开源组织,北京,中国)
Federated Heterogeneous Language Model Optimization for Hybrid Automatic Speech Recognition
Hong Kong Polytechnic University(香港理工大学);Ant Group(蚂蚁集团);Shanghai Jiao Tong University(上海交通大学);Hong Kong University of Science and Technology(香港科学与技术大学);Shenzhen University(深圳大学)