arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.01623cs.LGcs.AI
收藏

Fira:我们能否在低秩约束下实现大语言模型的全秩训练?

Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?

Beijing Institute of Technology(北京理工大学);School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);MMLab, The Chinese University of Hong Kong(香港中文大学MMLab);Hebei Province Key Laboratory of Big Data Science and Intelligent Technology(河北省大数据科学与智能技术重点实验室)

Xi Chen, Kaituo Feng, Changsheng Li, Xunhao Lai, Xiangyu Yue, Ye Yuan, Guoren Wang

AI总结:

针对现有低秩训练方法因约束在低秩子空间导致性能次优的问题,本文提出即插即用框架Fira,通过基于范数的缩放和范数增长限制器,在低秩内存约束下实现大语言模型全秩训练,性能优于LoRA、GaLore,可比肩全秩训练。

摘要:

低秩训练已成为降低大语言模型(LLM)训练内存占用的极具潜力的方法。此前的方法要么依赖权重矩阵分解(如LoRA),要么尝试分解梯度矩阵(如GaLore)来减少内存消耗。但这两类方法都将训练约束在低秩子空间内,不可避免地导致性能次优。这引出了一个问题:我们能否在始终保留低秩约束以保障内存效率的同时,实现全秩训练(即使用全秩权重的全秩梯度开展训练),从而避免性能欠佳的结果?本文提出了一款名为Fira的新型大语言模型即插即用训练框架,这是实现上述目标的首次尝试。首先,我们观察到大语言模型训练过程中的一个有趣现象:自适应优化器(如Adam)对梯度范数的缩放影响,在低秩训练到全秩训练的过程中保持相近。基于这一观察,我们提出了基于范数的缩放方法,该方法利用低秩优化器的缩放影响替代原始全秩优化器的缩放影响,以支撑全秩训练。通过这种方式,我们可以在优化器中保留低秩约束,同时实现全秩训练以获得更优性能。此外,我们发现优化过程中存在梯度骤升现象,可能导致损失尖峰。为解决这一问题,我们进一步提出了范数增长限制器,通过调控梯度范数的相对增幅来平滑梯度。在大语言模型预训练和微调任务上开展的大量实验表明,Fira的表现优于LoRA和GaLore,达到了与全秩训练相当甚至更优的性能。

英文摘要:

Low-rank training has emerged as a promising approach for reducing memory usage in training Large Language Models (LLMs). Previous methods either rely on decomposing weight matrices (e.g., LoRA), or seek to decompose gradient matrices (e.g., GaLore) to ensure reduced memory consumption. However, both of them constrain the training in a low-rank subspace, thus inevitably leading to sub-optimal performance. This raises a question: whether it is possible to consistently preserve the low-rank constraint for memory efficiency, while achieving full-rank training (i.e., training with full-rank gradients of full-rank weights) to avoid inferior outcomes? In this paper, we propose a new plug-and-play training framework for LLMs called Fira, as the first attempt to achieve this goal. First, we observe an interesting phenomenon during LLM training: the scaling impact of adaptive optimizers (e.g., Adam) on the gradient norm remains similar from low-rank to full-rank training. Based on this observation, we propose a norm-based scaling method, which utilizes the scaling impact of low-rank optimizers as substitutes for that of original full-rank optimizers to enable full-rank training. In this way, we can preserve the low-rank constraint in the optimizer while achieving full-rank training for better performance. Moreover, we find that there are sudden gradient rises during the optimization process, potentially causing loss spikes. To address this, we further put forward a norm-growth limiter to smooth the gradient via regulating the relative increase of gradient norms. Extensive experiments on the pre-training and fine-tuning of LLMs show that Fira outperforms both LoRA and GaLore, achieving performance that is comparable to or even better than full-rank training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑