arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.09758cs.LGcs.CL
收藏

BiDoRA:基于双层优化的权重分解低秩适配

BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation

University of California, San Diego(加州大学圣迭戈分校)

Peijia Qin, Ruiyi Zhang, Pengtao Xie

AI总结:

针对DoRA同步优化导致的过拟合和耦合更新问题,提出基于双层优化的BiDoRA,通过异步解耦优化幅度与方向,提升微调效果,在多项任务上优于现有方法。

摘要:

参数高效微调(PEFT)是一种灵活高效的方法,用于将大型语言模型(LLMs)适配到下游任务。在这些方法中,权重分解低秩适配(DoRA)是一种有前景的方法,它将权重矩阵分解为幅度和方向两个组成部分,以更好地模拟全量微调(FT)。然而,DoRA对这些组成部分的同步优化使其过度表达,增加了过拟合的风险,并形成了耦合的更新模式,限制了其学习能力。为解决这些问题,我们提出了基于双层优化的权重分解低秩适配(BiDoRA),一种基于双层优化框架的新型PEFT方法。BiDoRA与DoRA的根本区别在于,它使用不同的训练和验证数据划分,在两个独立的异步循环中分别优化幅度和方向。这种解耦的优化过程有效缓解了过拟合,并允许更灵活的更新,使其与全量微调更加一致。例如,权重分解分析显示,BiDoRA的幅度-方向更新相关性为-8.042,比DoRA的-1.784更接近全量微调的理想值。在涵盖自然语言理解、生成、词元分类和极小生物医学数据集等多种任务上的评估表明,BiDoRA始终优于DoRA以及众多领先的PEFT方法。这种改进具有统计显著性,如在GLUE基准上,通过Wilcoxon符号秩检验,BiDoRA以p值为2.4×10^-4超过了DoRA。BiDoRA的代码可在https://github.com/t2ance/BiDoRA获取。

英文摘要:

Parameter-efficient fine-tuning (PEFT) is a flexible and efficient method for adapting large language models (LLMs) to downstream tasks. Among these methods, weight-decomposed low-rank adaptation (DoRA) is a promising approach that decomposes weight matrices into magnitude and direction components to mimic full fine-tuning (FT) better. However, DoRA's simultaneous optimization of these components makes it over-expressive, increases the risk of overfitting, and creates a coupled updating pattern that limits its learning capacity. To address these issues, we propose Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation (BiDoRA), a novel PEFT method based on a bi-level optimization framework. BiDoRA fundamentally differs from DoRA by optimizing the magnitude and direction in two separate, asynchronous loops using distinct training and validation data splits. This decoupled optimization process effectively mitigates overfitting and allows for more flexible updates that align even more closely with FT. For instance, weight decomposition analysis shows BiDoRA achieves a magnitude-direction update correlation of $-8.042$, significantly closer to the FT ideal compared to $-1.784$ for DoRA. Evaluation of BiDoRA on diverse tasks spanning natural language understanding, generation, token classification, and extremely small biomedical datasets reveals that it consistently outperforms DoRA and a wide range of leading PEFT methods. This improvement is statistically significant, as demonstrated on the GLUE benchmark where BiDoRA surpasses DoRA with a p-value of $2.4\times10^{-4}$ in terms of the Wilcoxon signed-rank test. The code for BiDoRA is available at https://github.com/t2ance/BiDoRA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑