arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03237cs.LGcs.CL
收藏

超越合并:通过激活引导的旋转进行流式LLM更新

Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations

City University of Hong Kong, Hong Kong SAR(香港城市大学);Tsinghua University(清华大学);Huawei Noah’s Ark Lab, Hong Kong SAR(华为诺亚实验室(香港));University of Hong Kong(香港大学);Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州))

Yuxuan Yao, Haonan Sheng, Qingsong Lv, Han Wu, Shuqi Liu, Zehua Liu, Zengyan Liu, Jiahui Gao, Haochen Tan, Xiaojin Fu, Haoli Bai, Hing Cheung So, Zhijiang Guo, Linqi Song

AI总结:

本文提出ARM策略,通过激活引导的旋转实现流式LLM更新,有效超越收敛模型,提供高效适应框架。

摘要:

大型语言模型(LLMs)规模的不断扩大要求高效的适应技术。模型合并因其效率和可控性而受到关注。然而,现有的合并技术通常作为事后优化或侧重于减轻任务干扰,往往未能捕捉到监督微调(SFT)的动态优化优势。在本工作中,我们提出了流式合并,一种创新的模型更新范式,将合并视为迭代优化过程。该范式的核心是ARM(激活引导的旋转感知合并),一种旨在近似梯度下降动态的策略。通过将合并系数视为学习率,并从激活子空间中推导出旋转向量,ARM有效地将参数更新引导到数据驱动的轨迹上。与传统的线性插值不同,ARM对齐语义子空间以保持高维参数演化的几何结构。值得注意的是,ARM仅需要早期SFT检查点,并通过迭代合并超越完全收敛的SFT模型。在模型规模(1.7B到14B)和多样领域(如数学、代码)上的实验结果表明,ARM能够超越收敛检查点。广泛的实验表明,ARM提供了一种可扩展且轻量的框架,用于高效的模型适应。

英文摘要:

The escalating scale of Large Language Models (LLMs) necessitates efficient adaptation techniques. Model merging has gained prominence for its efficiency and controllability. However, existing merging techniques typically serve as post-hoc refinements or focus on mitigating task interference, often failing to capture the dynamic optimization benefits of supervised fine-tuning (SFT). In this work, we propose Streaming Merging, an innovative model updating paradigm that conceptualizes merging as an iterative optimization process. Central to this paradigm is \textbf{ARM} (\textbf{A}ctivation-guided \textbf{R}otation-aware \textbf{M}erging), a strategy designed to approximate gradient descent dynamics. By treating merging coefficients as learning rates and deriving rotation vectors from activation subspaces, ARM effectively steers parameter updates along data-driven trajectories. Unlike conventional linear interpolation, ARM aligns semantic subspaces to preserve the geometric structure of high-dimensional parameter evolution. Remarkably, ARM requires only early SFT checkpoints and, through iterative merging, surpasses the fully converged SFT model. Experimental results across model scales (1.7B to 14B) and diverse domains (e.g., math, code) demonstrate that ARM can transcend converged checkpoints. Extensive experiments show that ARM provides a scalable and lightweight framework for efficient model adaptation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑