arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2611.06486cs.LGcs.AI
收藏

模型合并无需统一LoRA秩

Model Merging Does Not Require Uniform LoRA Ranks

Amazon(亚马逊);Amazon Web Services(亚马逊网络服务)

Zirong Xie, Zainab Afolabi, Nikita Kozodoi, Jack Butler

AI总结:

该研究指出模型合并无需统一LoRA秩,为各专家分配不同秩可提升多领域准确率,而按损失影响选择分量的合并方法能保留所有专家并稳定准确率。

摘要:

模型合并是指将分别微调的专家模型组合成单个模型而无需重新训练的技术。标准做法是为每个专家模型训练相同的LoRA秩,这使得秩的选择在不同专家间相互关联,尽管每个领域都有其自身的最优秩。本文表明这种关联是合并方法的特性,而非合并本身的固有属性。我们在三个领域(数学、指令跟随、多语言)中,针对四个不同的秩和三种模型规模训练了36个专家模型,随后使用六种合并方法,分别在统一秩和各领域自身最优秩的情况下进行合并。为专家模型分配不同秩不会增加平均计算量和基于效用的选择的成本,且能使多领域准确率提升最多1.8个百分点;而冲突解决方法的表现并不稳定,可能导致准确率下降最多4个百分点。这种差异有一个简单的原因:合并方法仅保留池化谱分量的固定比例,按大小选择分量的方法会丢弃许多本可降低损失的分量,且倾向于丢弃同一专家模型——指令适配器的奇异值最小,因此几乎被完全丢弃,导致IFEval准确率下降16至25个百分点。而按对损失的影响选择分量的方法会保留所有三个专家模型,且在8倍秩范围内准确率几乎保持稳定(仅波动0.7个百分点,而4B规模下TIES方法的波动为6.7个百分点)。因此,只要合并算子不按大小选择分量,就可以为每个专家模型单独选择秩。

英文摘要:

Model merging combines separately fine-tuned experts into a single model without retraining. Standard practice trains every expert at the same LoRA rank, which couples the rank choice across experts even though each domain has its own best rank. We show this coupling is a property of the merge method, not of merging. We train 36 experts on three domains (mathematics, instruction following, multilingual) at four ranks and three model scales, then merge with six methods at uniform ranks and at each domain's own best rank. Giving experts different ranks costs averaging and utility-based selection nothing, and gains them up to 1.8 pp of multi-domain accuracy, while the conflict-resolution methods are inconsistent and can lose up to 4 pp. The split has a simple cause. Merging keeps only a fixed share of the pooled spectral components, and methods that choose them by size discard many that would have reduced loss. They also tend to discard the same expert: the instruction adapter has the smallest singular values, so it is dropped almost entirely and IFEval falls by 16 to 25 pp. Choosing components by their effect on loss instead keeps all three experts, and leaves accuracy nearly flat across an 8$\times$ rank range (0.7 pp, against 6.7 pp for TIES at 4B scale). Rank can therefore be chosen per expert whenever the merge operator does not select by magnitude.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑