arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2306.01708cs.LGcs.AIcs.CLcs.CV
收藏

TIES-Merging:解决模型合并时的干扰问题

TIES-Merging: Resolving Interference When Merging Models

University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);IBM Research(IBM研究院);MIT(麻省理工学院)

Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, Mohit Bansal

AI总结:

本文提出 TIES-Merging 方法,通过重置微调中变化较小的参数、解决符号冲突并仅合并符号一致的参数,有效缓解多模型合并时的参数干扰,在多种设置下优于现有方法。

摘要:

迁移学习——即在下游任务上进一步微调预训练模型——可以带来显著优势,包括提升下游性能、加快收敛速度以及提高样本效率。这些优势导致了大量任务特定微调模型的涌现,而这些模型通常只能执行单一任务,彼此之间无法相互受益。最近,模型合并技术作为一种解决方案出现,旨在将多个任务特定模型组合成一个多任务模型,而无需进行额外训练。然而,现有的合并方法往往忽略了不同模型参数之间的干扰,导致在合并多个模型时性能大幅下降。在本文中,我们证明了先前的合并技术由于两大干扰来源而无意中丢失了有价值的信息:(a)冗余参数值造成的干扰,以及(b)不同模型之间对给定参数值的符号存在分歧。为了解决这一问题,我们提出了我们的方法——TRIM, ELECT SIGN & MERGE(TIES-Merging),该方法在合并模型时引入了三个新步骤:(1)重置在微调过程中仅发生少量变化的参数,(2)解决符号冲突,以及(3)仅合并与最终达成一致的符号相一致的参数。我们发现,TIES-Merging 在涵盖多种模态、领域、任务数量、模型规模、架构和微调设置的多样化场景中,优于多种现有方法。我们进一步分析了不同类型的干扰对模型参数的影响,并强调了解决符号干扰的重要性。我们的代码可在 https://github.com/prateeky2806/ties-merging 获取。

英文摘要:

Transfer learning - i.e., further fine-tuning a pre-trained model on a downstream task - can confer significant advantages, including improved downstream performance, faster convergence, and better sample efficiency. These advantages have led to a proliferation of task-specific fine-tuned models, which typically can only perform a single task and do not benefit from one another. Recently, model merging techniques have emerged as a solution to combine multiple task-specific models into a single multitask model without performing additional training. However, existing merging methods often ignore the interference between parameters of different models, resulting in large performance drops when merging multiple models. In this paper, we demonstrate that prior merging techniques inadvertently lose valuable information due to two major sources of interference: (a) interference due to redundant parameter values and (b) disagreement on the sign of a given parameter's values across models. To address this, we propose our method, TRIM, ELECT SIGN & MERGE (TIES-Merging), which introduces three novel steps when merging models: (1) resetting parameters that only changed a small amount during fine-tuning, (2) resolving sign conflicts, and (3) merging only the parameters that are in alignment with the final agreed-upon sign. We find that TIES-Merging outperforms several existing methods in diverse settings covering a range of modalities, domains, number of tasks, model sizes, architectures, and fine-tuning settings. We further analyze the impact of different types of interference on model parameters, and highlight the importance of resolving sign interference. Our code is available at https://github.com/prateeky2806/ties-merging

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑