Model merging offers a promising avenue for knowledge integration and parallel development without retraining. Yet, existing methods either ignore the geometry of the loss landscape or rely on intractable full-space Hessian approximations. We propose EpiMer, a framework that casts model merging as solving the Fréchet mean on a Riemannian manifold and restricts the computation to a low-rank subspace spanned by the task vectors. With the expected Hessian as the metric, we reveal a connection between local curvature and epistemic uncertainty of the parameters. Our theoretical analysis decomposes the merging error bound into the subspace Fréchet variance and the residual energy, and provides a closed-form characterization of when curvature-aware merging provably outperforms flat-geometry methods. In addition, our framework unifies both curvature-aware methods and recent spectral methods as special cases of the subspace Fréchet mean with different geometric metrics. Merging fine-tuned CLIP-ViT models on eight image classification tasks, Epistemic Merging strictly outperforms the baselines on all three CLIP-ViT backbones at matched rank, improving the across-task average accuracy and worst-task accuracy on every backbone.
How does the optimizer implicitly bias the model merging loss landscape?
University of Luxembourg(卢森堡大学);ETH Zürich, MPI for Intelligent Systems(苏黎世联邦理工学院、智能系统研究所);Idiap Research Institute(Idiap研究机构);ELLIS Institute Tübingen, MPI for Intelligent Systems, Tübingen AI Center(图宾根ELLIS研究所、智能系统研究所、图宾根人工智能中心)
Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes
Peking University(北京大学);Beijing Institute of Big Data Research(北京大数据研究院);Center for Data Science, Peking University(北京大学数据科学中心);BICMR, Peking University(北京大学北京国际数学研究中心);Princeton University(普林斯顿大学)
Symmetry-Aware Graph Metanetwork Autoencoders: Model Merging through Parameter Canonicalization
University of Amsterdam(阿姆斯特丹大学);Informatics Institute, University of Amsterdam(阿姆斯特丹大学信息学院);Amsterdam Machine Learning Lab (AMLab), University of Amsterdam(阿姆斯特丹大学机器学习实验室)
When Muon Meets Task Interference: A Spectral Perspective on Continual Learning and Model Merging
State Key Laboratory for Novel Software Technology, Nanjing University(南京大学现代国家重点实验室(注:此处按原文直译,实际常见译名为“南京大学计算机软件新技术国家重点实验室”));University of Wollongong(卧龙岗大学)
School of Computation, Information and Technology, Technical University of Munich(计算、信息与技术学院,慕尼黑技术大学);Helmholtz Munich(海德堡-慕尼黑研究中心);Tübingen AI Center, University of Tübingen(图宾根人工智能中心,图宾根大学)
ChainLoRA: Geometry-Preserving Task Vector Merging for Continual Learning in LLMs
Shanghai Jiao Tong University(上海交通大学);Shanghai Innovation Institute(上海创新研究院);Beijing Zhongguancun Academy(北京中关村学院);StepFun(阶跃星辰);Huazhong University of Science and Technology(华中科技大学);IAAR(智能自动化研究院)
Extra-Merge: Tracing the Rank-1 Subspace of Model Merging in Language Model Pre-Training
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences, Beijing, China(中国科学院大学先进交叉学科学院);State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, China(中国科学院人工智能安全国家重点实验室);University of Chinese Academy of Sciences, China(中国科学院大学);Alibaba Group, China(阿里巴巴集团);School of Mathematics, Southeast University, Nanjing, China(东南大学数学学院)
Tsinghua University(清华大学);PCL Research Center of Networks and Communications, Peng Cheng Laboratory(鹏城实验室网络通信PCL研究中心);Key Lab. of Machine Perception (MoE), School of EECS, Peking University(北京大学信息科学技术学院机器感知教育部重点实验室)