大语言模型（LLMs）合并旨在将多个同源 LLM 融合为一个具备全部能力的模型。理想情况下，任何共享相同骨干网络的 LLM 都应当可合并，无论它们是参数变化较小的微调（FT）模型，还是参数偏移较大的预训练（PT）模型。然而，现有方法通常手动指定模型重要性，使其仅适用于参数变化相似的 LLM，例如多个 FT LLM。FT 与 PT LLM 之间不同的参数变化范围给当前解决方案在经验上确定最优组合带来了挑战。在本文中，我们做出了开创性努力，将合并技术的适用范围从 FT LLM 扩展到 PT LLM。我们首先考察了当前方法在合并 FT 与 PT LLM 时的有效性，发现它们难以处理 PT LLM。随后，我们提出了一种基于权重解耦（WIDEN）的方法来有效扩展合并范围，该方法首先将模型权重解耦为幅值和方向两个分量，然后根据它们各自的贡献进行自适应融合。在实验中，我们在 7B 和 14B 模型规模上合并了 Qwen1.5-Chat（具备指令跟随能力的 FT LLM）与 Sailor（具备多语言能力的 PT LLM）。结果表明：（1）现有解决方案在合并 Sailor 时通常会失败，要么同时丧失两种能力，要么仅保留指令跟随能力；（2）WIDEN 成功将 Sailor 的多语言能力注入 Qwen1.5-Chat，使其精通东南亚语言，并在基础能力上取得提升。结合先前研究，我们还合并了多个 13B FT LLM，并观察到 WIDEN 在指令跟随、数学推理和代码生成能力上实现了均衡融合。
英文摘要：
Merging Large Language Models (LLMs) aims to amalgamate multiple homologous LLMs into one with all the capabilities. Ideally, any LLMs sharing the same backbone should be mergeable, irrespective of whether they are Fine-Tuned (FT) with minor parameter changes or Pre-Trained (PT) with substantial parameter shifts. However, existing methods often manually assign the model importance, rendering them feasible only for LLMs with similar parameter alterations, such as multiple FT LLMs. The diverse parameter changed ranges between FT and PT LLMs pose challenges for current solutions in empirically determining the optimal combination. In this paper, we make a pioneering effort to broaden the applicability of merging techniques from FT to PT LLMs. We initially examine the efficacy of current methods in merging FT and PT LLMs, discovering that they struggle to deal with PT LLMs. Subsequently, we introduce an approach based on WeIght DisENtanglement (WIDEN) to effectively extend the merging scope, which first disentangles model weights into magnitude and direction components, and then performs adaptive fusion by considering their respective contributions. In the experiments, we merge Qwen1.5-Chat (an FT LLM with instruction-following skills) with Sailor (a PT LLM with multilingual abilities) across 7B and 14B model scales. Results reveal that: (1) existing solutions usually fail when merging Sailor, either losing both abilities or only retaining instruction-following skills; (2) WIDEN successfully injects the multilingual abilities of Sailor into Qwen1.5-Chat and make it proficient in Southeast Asian languages, achieving enhancements in the fundamental capabilities. In light of previous research, we also merge multiple 13B FT LLMs and observe that WIDEN achieves a balanced amalgamation of instruction following, mathematical reasoning, and code generation skills.
Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea（POSTECH人工智能研究生院，韩国）；Department of Computer Science and Engineering, POSTECH, Republic of Korea（POSTECH计算机科学与工程系，韩国）
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
School of Computer Science & Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL)（华中科技大学与平安产险联合实验室）；Ping An Property & Casualty Insurance Company of China, Ltd.（中国平安财产保险股份有限公司）；The Chinese University of Hong Kong（香港中文大学）
Personality Vector: Modulating Personality of Large Language Models by Model Merging
Department of Human-Artificial Intelligence Interaction, Sungkyunkwan University（人类-人工智能交互系，成均馆大学）；Department of Immersive Media Engineering, Sungkyunkwan University（沉浸式媒体工程系，成均馆大学）
City University of Hong Kong（香港城市大学）；Tencent Jarvis Lab（腾讯 Jarvis 实验室）；University of Technology Sydney（悉尼大学）；University of Queensland（昆士兰大学）；Jilin University（吉林大学）；Westlake University（西湖大学）
Activation-Guided Consensus Merging for Large Language Models
Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）；Huawei Noah’s Ark Lab, Hong Kong SAR（华为诺亚实验室（香港特别行政区））；University of Hong Kong（香港大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））
Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging
University of Florida（佛罗里达大学）；Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida（佛罗里达大学医学院健康结果与生物医学信息学系）；Department of Engineering Education, Herbert Wertheim College of Engineering, University of Florida（佛罗里达大学赫伯特·韦特海姆工程学院工程教育系）；Preston A. Wells, Jr. Center for Brain Tumor Therapy, Lillian S. Wells Department of Neurosurgery, University of Florida（佛罗里达大学莉莉安·S·威尔斯神经外科系普雷斯顿·A·威尔斯脑肿瘤治疗中心）
AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging
DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；National University of Singapore（新加坡国立大学）；Hupan Lab（湖畔实验室）；Singapore University of Technology and Design（新加坡科技设计大学）
Activation-Informed Merging of Large Language Models
Massachusetts Institute of Technology（麻省理工学院）；Stony Brook University（石溪大学）；MIT-IBM Watson AI Lab & Red Hat AI Innovation（MIT-IBM Watson AI实验室及Red Hat AI创新）
PatientDx: Merging Large Language Models for Protecting Data-Privacy in Healthcare
Université de Toulouse（图卢兹大学）；IRIT UMR 5505；Centre Hospitalier Universitaire de Toulouse（图卢兹大学医院中心）；INSERM（法国国家健康与医学研究院）；Faculté de Médecine Université de Toulouse（图卢兹大学医学院）
Multi-Modality Expansion and Retention for LLMs through Parameter Merging and Decoupling
Harbin Institute of Technology, Shenzhen, China（哈尔滨工业大学（深圳））；Xiamen University Malaysia（厦门大学马来西亚分校）；Nanyang Technological University（南洋理工大学）；Beijing Academy of Artificial Intelligence, China（北京人工智能研究院）；Peng Cheng Laboratory, China（鹏城实验室）；Shaqra University, Saudi Arabia（沙迦大学）
RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format
Institute of Image Processing and Pattern Recoginition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（图像处理与模式识别研究所，自动化与智能感知学院，上海交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；MoE Key Laboratory of System Control and Information Processing (Shanghai)（系统控制与信息处理国家重点实验室（上海））
AP-BMM: Approximating Capability-Cost Pareto Sets of LLMs via Asynchronous Prior-Guided Bayesian Model Merging
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东新型安全智能技术重点实验室）；Institute of Cyberspace Security（网络空间安全研究院）；School of Computer Science and Technology（计算机科学与技术学院）
Mediator: Memory-efficient LLM Merging with Less Parameter Conflicts and Uncertainty Based Routing
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Platform and Content Group, Tencent（腾讯平台与内容事业群）；UCAS（中国科学院大学）；Sun Yat-sen University（中山大学）
Will it Merge? On The Causes of Model Mergeability
Technion - Israel Institute of Technology（技术ion-以色列理工学院）；IBM Research AI（IBM人工智能研究部）；MIT（麻省理工学院）；MIT-IBM Watson AI Lab（麻省理工-IBM沃森人工智能实验室）；Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Kempner Institute, Harvard University（哈弗大学凯普纳研究所）
GIFT: Guided Fine-Tuning and Transfer for Enhancing Instruction-Tuned Language Models
Southern University of Science and Technology（南方科技大学）；Alibaba Group（阿里巴巴集团）；Shanghai University of Finance and Economics（上海财经大学）；Tsinghua University（清华大学）；Xiamen University（厦门大学）
