arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.14538cs.CL
收藏

LoRA-MGPO:通过动量引导的扰动优化缓解低秩适应中的双重下降

LoRA-MGPO: Mitigating Double Descent in Low-Rank Adaptation via Momentum-Guided Perturbation Optimization

School of Artificial Intelligence, Jilin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China(知识驱动人机智能工程研究中心,教育部,中国);International Center of Future Science, Jilin University(未来科学国际中心,吉林大学)

Yupeng Chang, Chenlu Guo, Yi Chang, Yuan Wu

AI总结:

针对LoRA随秩增加出现的双重下降问题,提出LoRA-MGPO框架,利用动量引导扰动优化和自适应归一化稳定训练,在NLU和生成基准上优于现有PEFT方法。

摘要:

参数高效微调(PEFT),特别是低秩适应(LoRA),通过仅训练一小部分参数来适应大型语言模型(LLM)。然而,随着用于适应的低秩矩阵的秩增加,LoRA 常常表现出不稳定的“双重下降”现象,其特征是训练损失的瞬时发散,这会延迟收敛并因吸引到尖锐局部最小值而导致不稳定性,从而损害泛化能力。为了解决这个问题,我们引入了 LoRA-MGPO,一个结合了动量引导的扰动优化(MGPO)的框架。MGPO 通过缓解双重下降现象并利用优化器状态中的动量向量来引导权重扰动,从而稳定训练动态,避免了双重梯度计算。此外,一种自适应归一化方案基于梯度范数的指数移动平均(EMA)来缩放扰动的幅度,进一步增强了稳定性。虽然 EMA 控制扰动的幅度,MGPO 引导其方向,确保更稳定的优化轨迹。在一系列自然语言理解和生成基准上的实验表明,LoRA-MGPO 始终优于 LoRA 和其他 PEFT 方法。分析表明,LoRA-MGPO 通过稳定训练过程和缓解对尖锐最小值的吸引,实现了更平滑的损失曲线、更快的收敛和更好的泛化。

英文摘要:

Parameter-efficient fine-tuning (PEFT), particularly Low-Rank Adaptation (LoRA), adapts large language models (LLMs) by training only a small fraction of parameters. However, as the rank of the low-rank matrices used for adaptation increases, LoRA often exhibits an unstable "double descent" phenomenon, characterized by transient divergence in the training loss, which delays convergence and impairs generalization by causing instability due to the attraction to sharp local minima. To address this, we introduce LoRA-MGPO, a framework that incorporates Momentum-Guided Perturbation Optimization (MGPO). MGPO stabilizes training dynamics by mitigating the double descent phenomenon and guiding weight perturbations using momentum vectors from the optimizer's state, thus avoiding dual gradient computations. Additionally, an adaptive normalization scheme scales the magnitude of perturbations based on an exponential moving average (EMA) of gradient norms, further enhancing stability. While EMA controls the magnitude of the perturbations, MGPO guides their direction, ensuring a more stable optimization trajectory. Experiments on a suite of natural language understanding and generation benchmarks show that LoRA-MGPO consistently achieves superior performance over LoRA and other PEFT methods. The analysis indicates that LoRA-MGPO leads to smoother loss curves, faster convergence, and improved generalization by stabilizing the training process and mitigating the attraction to sharp minima.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑