模型合并旨在以低成本将多个独立的特定任务模型组合为单个多任务模型。在本研究中,我们将以往的合并方法视为利用不同的“任务参数子空间”概念——模型在合并前会在该子空间中进行匹配。我们将给定模型的任务参数子空间与其损失景观相关联,并形式化阐述了这种模型合并方法可被视为求解一个线性方程组。以往的研究通常局限于存在闭式解的线性系统,而我们考虑使用共轭梯度法来求解。我们表明,使用共轭梯度法的效果优于闭式解,能够通过原本难以求解的线性系统实现合并,并且可以灵活选择多种初始化方式和“任务参数子空间”的估计方法。我们最终证明,名为“在任务参数子空间中匹配模型”(Matching Models in their Task Parameter Subspace, MaTS)的合并框架在多任务和中间任务模型合并任务上取得了当前最优结果。我们已将本研究使用的所有代码和检查点发布在https://github.com/r-three/mats。
英文摘要:
Model merging aims to cheaply combine individual task-specific models into a single multitask model. In this work, we view past merging methods as leveraging different notions of a ''task parameter subspace'' in which models are matched before being merged. We connect the task parameter subspace of a given model to its loss landscape and formalize how this approach to model merging can be seen as solving a linear system of equations. While past work has generally been limited to linear systems that have a closed-form solution, we consider using the conjugate gradient method to find a solution. We show that using the conjugate gradient method can outperform closed-form solutions, enables merging via linear systems that are otherwise intractable to solve, and flexibly allows choosing from a wide variety of initializations and estimates for the ''task parameter subspace''. We ultimately demonstrate that our merging framework called ''Matching Models in their Task Parameter Subspace'' (MaTS) achieves state-of-the-art results in multitask and intermediate-task model merging. We release all of the code and checkpoints used in our work at https://github.com/r-three/mats.
Bridging Domains through Subspace-Aware Model Merging
Universidade Estadual de Campinas (UNICAMP)(科林斯州立大学);Recod.ai Lab.(Recod.ai实验室);Instituto de Computação(计算研究所);CISPA Helmholtz Center for Information Security(信息安全赫尔姆霍茨中心);Intercom
School of Computer Science and Engineering, Southeast University, Nanjing, China(东南大学计算机科学与工程学院,南京,中国);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其跨学科应用国家重点实验室(东南大学),中华人民共和国,中国);Huawei Technologies, Shanghai, China(华为技术有限公司,上海,中国)
School of Computation, Information and Technology, Technical University of Munich(计算、信息与技术学院,慕尼黑技术大学);Helmholtz Munich(海德堡-慕尼黑研究中心);Tübingen AI Center, University of Tübingen(图宾根人工智能中心,图宾根大学)
No Task Left Behind: Isotropic Model Merging with Common and Task-Specific Subspaces
Warsaw University of Technology, Poland;IDEAS Research Center, Warsaw, Poland;Department of Information Engineering, Media Integration;Communication Center (MICC), University of Florence, Italy;NASK - PIB, National Research Institute, Warsaw, Poland;Gdańsk University of Technology, Poland;Computer Vision Center, Barcelona, Spain;Department of Computer Science, Universitat Autonoma de Barcelona, Spain
FRISM: Fine-Grained Reasoning Injection via Subspace-Level Model Merging for Vision-Language Models
College of Future Information Technology, Fudan University, Shanghai, China(复旦大学未来信息科技学院,中国);Shanghai Innovation Institute, China(上海创新研究院,中国);The Chinese University of Hong Kong, China(香港中文大学,中国);Harbin Institute of Technology, China(哈尔滨工业大学,中国);Shanghai Artificial Intelligence Laboratory, China(上海人工智能实验室,中国);Sun Yat-Sen University, Shenzhen, China(暨南大学深圳校区,中国)
Data-Aware Model Merging in the Essential Subspace for Multi-Task Knowledge Aggregation
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其交叉应用国家重点实验室(东南大学));Huawei Inc.(华为公司)
Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China(中国科学院大学先进交叉学科学院);State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, China(中国科学院人工智能安全国家重点实验室);University of Chinese Academy of Sciences, China(中国科学院大学);Alibaba Group, China(阿里巴巴集团);School of Mathematics, Southeast University, Nanjing, China(东南大学数学学院)
Model Merging by Uncertainty-Based Gradient Matching
Technical University of Darmstadt(达姆施塔特工业大学);Hessian Center for AI (hessian.AI)(黑森人工智能中心);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心);University of Edinburgh(爱丁堡大学)
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
School of Computer Science, Wuhan University(武汉大学计算机学院);Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Sun Yat-sen University(中山大学);Alibaba Group(阿里巴巴集团);School of Information and Electronics, Beijing Institute of Technology(北京理工大学信息与电子学院);Nanyang Technological University(南洋理工大学)
Merging Models on the Fly Without Retraining: A Sequential Approach to Scalable Continual Model Merging
Wuhan University(武汉大学);Northeastern University(东北大学);Sun Yat-sen University(中山大学);Beijing Institute of Technology(北京理工大学);Nanyang Technological University(南洋理工大学)
Generative Modeling of Discrete Data Using Geometric Latent Subspaces
Institute for Mathematics, Heidelberg University(海德堡大学数学研究所);Mannheim Institute for Intelligent Systems in Medicine, Heidelberg University(海德堡大学曼海姆智能医学系统研究所);IWR, Heidelberg University(海德堡大学IWR研究所);Research Station Geometry and Dynamics, Heidelberg University(海德堡大学几何与动力学研究站)
Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
Institute of Image Processing and Pattern Recognition, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China(图像处理与模式识别研究所,自动化与智能感知学院,上海交通大学,上海,中国);Shanghai Key Laboratory of Flexible Medical Robotics, Tongren Hospital, Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China(柔性医疗机器人上海市重点实验室,同仁医院,医疗机器人研究所,上海交通大学,上海,中国)
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
Shanghai Jiao Tong University(上海交通大学);Shanghai Innovation Institute(上海创新研究院);Beijing Zhongguancun Academy(北京中关村学院);StepFun(阶跃星辰);Huazhong University of Science and Technology(华中科技大学);IAAR(智能自动化研究院)
From Coefficients to Directions: Rethinking Model Merging with Directional Alignment
University of Oxford(牛津大学);Tsinghua University(清华大学);Nanyang Technological University(南洋理工大学);East China Normal University(华东师范大学);RIKEN(日本研究机构);The University of Tokyo(东京大学);Southern University of Science and Technology(南方科技大学)