arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2504.18580cs.LG
收藏

通过指标加权平均实现参数高效的检查点合并

Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging

Tensorplex Labs(Tensorplex实验室);Twelve Labs

Shi Jie Yu, Sehyun Choi

AI总结:

本文针对参数高效微调(PEFT)提出指标加权平均(MWA)方法,根据训练损失或步数对模型检查点进行加权合并,仅需一个超参数,实验表明该方法优于均匀合并基线,最高提升5%准确率。

摘要:

检查点合并是一种将多个模型快照组合成单一更优模型的技术,有可能减少大型语言模型的训练时间。本文探讨了参数高效微调(PEFT)背景下的检查点合并,其中仅训练小型适配器模块(例如 LoRA)。我们提出了指标加权平均(MWA),这是一种简单而有效的方法,通过根据性能指标对模型检查点的参数进行加权来合并它们。特别是,我们研究了通过训练损失和训练步数进行加权,基于直觉认为损失更低或步数更晚的检查点更有价值。我们引入了一个带有惩罚因子的公式来调整权重分布,无论检查点数量如何,只需一个超参数。在三个微调任务(数学推理、偏好对齐和通用指令微调)上的实验表明,MWA 一致地生成了优于朴素的检查点均匀平均的合并模型。值得注意的是,损失加权合并通常产生最佳结果,任务准确率比基线均匀合并高出 5%,甚至超过了最终单个检查点的性能。这些发现验证了 PEFT 的检查点合并,并证明了指标驱动的加权启发式方法可以以最小的计算开销有效提升模型性能。

英文摘要:

Checkpoint merging is a technique for combining multiple model snapshots into a single superior model, potentially reducing training time for large language models. This paper explores checkpoint merging in the context of parameter-efficient fine-tuning (PEFT), where only small adapter modules (e.g. LoRA) are trained. We propose Metrics-Weighted Averaging (MWA), a simple yet effective method to merge model checkpoints by weighting their parameters according to performance metrics. In particular, we investigate weighting by training loss and by training steps, under the intuition that lower-loss or later-step checkpoints are more valuable. We introduce a formula with a penalty factor to adjust weight distribution, requiring only one hyperparameter regardless of the number of checkpoints. Experiments on three fine-tuning tasks (mathematical reasoning, preference alignment, and general instruction tuning) show that MWA consistently produces merged models that outperform the naive uniform average of checkpoints. Notably, loss-weighted merging often yields the best results, delivering up to 5% higher task accuracy than the baseline uniform merge and even surpassing the final individual checkpoint's performance. These findings validate checkpoint merging for PEFT and demonstrate that a metric-driven weighting heuristic can efficiently boost model performance with minimal computational overhead.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑