arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2503.20110cs.CLcs.AIcs.LG
收藏

通过微调迁移实现高效模型开发

Efficient Model Development through Fine-tuning Transfer

Virginia Tech(弗吉尼亚理工大学);University of Toronto(多伦多大学);Vector Institute(向量研究所)

Pin-Jie Lin, Rishab Balasubramanian, Fengyuan Liu, Nikhil Kandpal, Tu Vu

AI总结:

本文提出通过迁移模型版本间的微调 diff 向量实现高效更新,无需额外训练即可显著提升目标基础模型性能,为持续大语言模型开发提供了低成本策略。

摘要:

现代大语言模型(LLMs)在高效更新方面面临困难,因为每个新的预训练模型版本都需要重复昂贵的对齐过程。这一挑战同样适用于领域特定或语言特定的模型,在这些模型中,针对专门数据的微调必须在每个新基础模型发布时重新进行。在本文中,我们探索了模型版本之间微调更新的迁移。具体来说,我们从一个源模型版本中导出 diff 向量(表示微调产生的权重变化),并将其应用于不同目标版本的基础模型。通过对各种开放权重模型版本的实证评估,我们表明迁移 diff 向量可以显著提升目标基础模型的性能。例如,将 Llama 3.0 8B 的微调更新迁移到 Llama 3.1 8B,在无需额外训练的情况下,在 IFEval 上提升了 46.9%,在 LiveCodeBench 上提升了 15.7%,甚至超过了 Llama 3.1 8B Instruct。此外,我们展示了在多语言任务上的性能增益,在马达加斯加语和土耳其语的 Global MMLU 上分别取得了 4.7% 和 15.5% 的提升。我们观察到,这些合并后的模型为进一步微调提供了更强的初始化。最后,我们的受控实验表明,当源模型和目标模型位于参数空间的线性连通区域时,微调迁移最为有效,并且我们对该方法进行了理论分析。综上所述,微调迁移为持续的大语言模型开发提供了一种成本高效且实用的策略。我们的代码可在 github.com/pjlintw/finetuning-transfer 获取。

英文摘要:

Modern LLMs struggle with efficient updates, as each new pretrained model version requires repeating expensive alignment processes. This challenge also applies to domain- or languagespecific models, where fine-tuning on specialized data must be redone for every new base model release. In this paper, we explore the transfer of fine-tuning updates between model versions. Specifically, we derive the diff vector (representing the weight changes from finetuning) from one source model version and apply it to the base model of a different target version. Through empirical evaluations on various open-weight model versions, we show that transferring diff vectors can significantly improve the performance of the target base model. For example, transferring the fine-tuning updates from Llama 3.0 8B improves Llama 3.1 8B by 46.9% on IFEval and 15.7% on LiveCodeBench without additional training, even surpassing Llama 3.1 8B Instruct. Furthermore, we demonstrate performance gains on multilingual tasks, with 4.7% and 15.5% improvements on Global MMLU for Malagasy and Turkish, respectively. We observe that these merged models provide stronger initializations for further fine-tuning. Lastly, our controlled experiments suggest that fine-tuning transfer is most effective when source and target models lie in a linearly connected region of parameter space, and we provide a theoretical analysis of our method. Taken together, fine-tuning transfer offers a cost-efficient and practical strategy for continuous LLM development. Our code is available at github.com/pjlintw/finetuning-transfer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑