Localize-and-Stitch：通过稀疏任务算术实现高效模型合并
Localize-and-Stitch: Efficient Model Merging via Sparse Task Arithmetic
AI总结：
本文提出 Localize-and-Stitch，通过定位仅占总参数 1% 的任务关键稀疏区域并将其拼回预训练模型，缓解全局模型合并的任务干扰，在视觉与语言基准上实现更优性能、压缩和持续技能组合。
摘要：
模型合并提供了一种有效策略，可将多个微调模型的优势结合到一个统一模型中，同时保留每个模型的专门能力。现有方法以全局方式合并模型，对所有模型参数执行算术运算。然而，这种全局合并通常会导致任务干扰，降低合并后模型的性能。在本工作中，我们提出 Localize-and-Stitch，这是一种以局部化方式合并模型的新颖方法。我们的算法分为两个步骤：i）局部化：在微调模型中识别极小的局部区域（占总参数的 1%），这些区域包含下游任务所需的关键技能；ii）拼接：仅将这些关键区域重新整合回预训练模型，以实现任务协同。我们证明，该方法能够有效定位负责微调性能的稀疏区域，并且这些局部化区域可被视为微调模型（任务）的紧凑且可解释的表示。在实验上，我们在多种视觉和语言基准上评估了该方法，结果表明其在不同数据可用场景下均优于现有模型合并方法。除了出色的实证性能外，我们的算法还有助于模型压缩并保留预训练知识，从而能够以极低的存储和计算开销，从多个微调模型中灵活、持续地组合技能。我们的代码可在 https://github.com/uiuctml/Localize-and-Stitch 获取。
英文摘要：
Model merging offers an effective strategy to combine the strengths of multiple finetuned models into a unified model that preserves the specialized capabilities of each. Existing methods merge models in a global manner, performing arithmetic operations across all model parameters. However, such global merging often leads to task interference, degrading the performance of the merged model. In this work, we introduce Localize-and-Stitch, a novel approach that merges models in a localized way. Our algorithm works in two steps: i) Localization: identify tiny ($1\%$ of the total parameters) localized regions in the finetuned models containing essential skills for the downstream tasks, and ii) Stitching: reintegrate only these essential regions back into the pretrained model for task synergy. We demonstrate that our approach effectively locates sparse regions responsible for finetuned performance, and the localized regions could be treated as compact and interpretable representations of the finetuned models (tasks). Empirically, we evaluate our method on various vision and language benchmarks, showing that it outperforms existing model merging methods under different data availability scenarios. Beyond strong empirical performance, our algorithm also facilitates model compression and preserves pretrained knowledge, enabling flexible and continual skill composition from multiple finetuned models with minimal storage and computational overhead. Our code is available at https://github.com/uiuctml/Localize-and-Stitch.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
