外推合并:通过外推与合并持续提升模型性能
Extrapolation Merging: Keep Improving With Extrapolation and Merging
AI总结:
本文提出外推合并范式,通过外推方法为模型合并提供明确优化方向,在无需额外计算资源和数据的情况下持续提升微调后模型性能,并在七个任务上验证了其有效性。
摘要:
大语言模型(LLMs)需要经过指令微调才能执行不同的下游任务。然而,指令微调阶段仍然需要大量的计算资源和标注数据,目前缺乏一种无需额外计算能力和数据即可提升模型性能的范式。模型合并旨在通过组合不同模型的参数来提升性能,但在合并过程中缺乏明确的优化方向,并不总能保证性能得到改善。本文尝试为模型合并提供一个明确的优化方向。我们首先验证了模型外推方法在指令微调阶段的有效性。随后,我们提出了外推合并(Extrapolation Merging),这是一种无需额外计算资源或数据即可持续提升模型性能的范式。利用外推方法,我们为模型合并提供了明确的方向,实现了局部优化搜索,从而提升了合并后模型的性能。我们在七个不同的任务上进行了实验,结果表明,我们的方法能够在微调后持续提升模型性能。
英文摘要:
Large Language Models (LLMs) require instruction fine-tuning to perform different downstream tasks. However, the instruction fine-tuning phase still demands significant computational resources and labeled data, lacking a paradigm that can improve model performance without additional computational power and data. Model merging aims to enhance performance by combining the parameters of different models, but the lack of a clear optimization direction during the merging process does not always guarantee improved performance. In this paper, we attempt to provide a clear optimization direction for model merging. We first validate the effectiveness of the model extrapolation method during the instruction fine-tuning phase. Then, we propose Extrapolation Merging, a paradigm that can continue improving model performance without requiring extra computational resources or data. Using the extrapolation method, we provide a clear direction for model merging, achieving local optimization search, and consequently enhancing the merged model's performance. We conduct experiments on seven different tasks, and the results show that our method can consistently improve the model's performance after fine-tuning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。