arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.08699cs.LG
收藏

缓解语言迁移中的灾难性遗忘:基于模型合并的方法

Mitigating Catastrophic Forgetting in Language Transfer via Model Merging

INSAIT, Sofia University “St. Kliment Ohridski”(INSAIT、索菲亚大学“圣·克利门特·奥赫里德斯基”);ETH Zurich(苏黎世联邦理工学院);University of Chicago(芝加哥大学);Together AI;Google DeepMind(谷歌DeepMind)

Anton Alexandrov, Veselin Raychev, Mark Niklas Müller, Ce Zhang, Martin Vechev, Kristina Toutanova

AI总结:

提出Branch-and-Merge(BaM)模型合并方法,通过迭代合并子集微调模型,在保加利亚语和德语上显著减少灾难性遗忘,同时匹配或提升目标语言性能。

摘要:

随着开放权重大型语言模型(LLMs)在英语的广泛任务中取得越来越令人瞩目的性能,实践者旨在将这些模型适配到不同语言。然而,这种语言适配常常伴随着基础模型能力的灾难性遗忘,严重限制了所得模型的实用性。我们通过提出Branch-and-Merge(BaM)来解决这一问题,这是一种基于迭代合并多个模型的新适配方法,这些模型在可用训练数据的子集上进行微调。BaM基于这样的洞察:这会产生幅度更低但质量更高的权重变化,在保持目标领域学习的同时减少对源领域的遗忘。我们在保加利亚语和德语上的广泛实证研究表明,与标准持续预训练和指令微调相比,BaM在不同模型架构上能显著减少遗忘,同时匹配甚至提升目标领域性能。

英文摘要:

As open-weight large language models (LLMs) achieve ever more impressive performances across a wide range of tasks in English, practitioners aim to adapt these models to different languages. However, such language adaptation is often accompanied by catastrophic forgetting of the base model's capabilities, severely limiting the usefulness of the resulting model. We address this issue by proposing Branch-and-Merge (BaM), a new adaptation method based on iteratively merging multiple models, fine-tuned on a subset of the available training data. BaM is based on the insight that this yields lower magnitude but higher quality weight changes, reducing forgetting of the source domain while maintaining learning on the target domain. We demonstrate in an extensive empirical study on Bulgarian and German that BaM can significantly reduce forgetting while matching or even improving target domain performance compared to both standard continued pretraining and instruction finetuning across different model architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑