PoLoRA：一种预处理正交化的LoRA优化器

中文摘要 AI 辅助

低秩适应（LoRA）通过向每个权重矩阵添加一个可训练的低秩更新来降低微调大语言模型的成本，该更新由两个矩阵的乘积参数化。这些矩阵通常用Adam训练，它将它们视为单个扁平参数向量，忽略了LoRA的矩阵和乘积结构。应用矩阵感知优化器（如Muon）对每个因子进行优化，并没有比Adam有持续的改进，同时工作中提出的乘积感知Muon变体也没有。为了实现持续的收益，我们引入了PoLoRA，一种预处理正交化的LoRA优化器，它由三个要素构建而成：乘积感知谱更新方向、通过控制每个样本的损失变化导出的曲率预处理，以及控制因子和合并更新大小的幅度规则。我们在从1B到8B参数的模型的代码和数学指令调优数据集上评估了PoLoRA，发现它在少1.2 - 1.7倍的步数内达到了调优Adam最终的保留损失，同时每步最多增加3%的开销。与Adam相比，PoLoRA对学习率也不太敏感，并且其最优学习率在不同秩上是稳定的。

英文摘要

Low-rank adaptation (LoRA) makes finetuning large language models cheaper by adding to each weight matrix a trainable low-rank update parameterized as the product of two matrices. These matrices are usually trained with Adam, which treats them as a single flat vector of parameters and ignores both the matrix and product structure of LoRA. Applying a matrix-aware optimizer such as Muon to each factor does not consistently improve over Adam, and neither do the product-aware Muon variants proposed in concurrent works. To realize consistent gains, we introduce PoLoRA, a Preconditioned Orthogonalized LoRA optimizer built from three ingredients: a product-aware spectral update direction, curvature preconditioning derived from controlling the per-sample loss change, and a magnitude rule that controls the sizes of both the factor and merged updates. We evaluate PoLoRA on instruction-tuning datasets for code and math across models from 1B to 8B parameters, and find that it reaches the final held-out loss achieved by tuned Adam in 1.2-1.7 times fewer steps, while adding at most 3% per-step overhead. Compared to Adam, PoLoRA is also less sensitive to the learning rate, and its optimal learning rate is stable across ranks.

发表机构

• Center for Computational Mathematics, Flatiron Institute（计算数学中心，熨斗研究院）

机构由 AI 辅助整理，请以论文原文为准。