谁引发干扰,谁就应终结它:通过任务向量引导无数据模型合并
Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors
AI总结:
针对无数据模型合并中的参数干扰问题,本文基于线性层任务向量构成近似线性子空间的理论发现,提出无需额外数据或重缩放系数的 WUDI-Merging,并在视觉与语言基准上取得SOTA性能。
摘要:
模型合并旨在将特定任务的专家模型整合到统一架构中,同时保持多任务泛化能力,但组成模型之间的参数干扰经常导致性能下降。尽管先前工作已经探索了许多合并策略,但在没有额外数据用于再训练或测试时计算的情况下解决干扰仍然具有挑战性。本文从理论上证明,线性层的任务向量为其对应输入构成一个近似线性子空间。因此,我们可以在任务向量的引导下最小化干扰。基于这一洞见,我们提出 WUDI-Merging(Whoever started the interference shoUld enD It,即“谁引发干扰,谁就应终结它”),这是一种简单而有效的模型合并方法,无需任何额外数据或重缩放系数即可消除干扰。跨视觉和语言基准的综合实证评估证明了本方法的优越性:在无数据模型合并场景中取得最先进性能(相比基线方法平均提升10.9%),甚至比主流测试时自适应方法高出3.3%,并且只需要极少的计算资源。代码将很快公开发布。
英文摘要:
Model merging seeks to integrate task-specific expert models into a unified architecture while preserving multi-task generalization capabilities, yet parameter interference between constituent models frequently induces performance degradation. Although prior work has explored many merging strategies, resolving interference without additional data for retraining or test-time computation remains challenging. In this paper, we theoretically demonstrate that the task vectors of the linear layer constitute an approximate linear subspace for its corresponding input. Therefore, we can minimize interference under the guidance of task vectors. Based on this insight, we propose \textbf{WUDI-Merging} (\textbf{W}hoever started the interference sho\textbf{U}ld en\textbf{D} \textbf{I}t), a simple yet effective model merging method that eliminates interference without any additional data or rescaling coefficients. Comprehensive empirical evaluations across vision and language benchmarks demonstrate our method's superiority, achieving state-of-the-art performance in data-free model merging scenarios (average 10.9\% improvement versus baseline methods) while even outperforming mainstream test-time adaptation approaches by 3.3\%, and only very few computing resources are required. The code will be publicly available soon.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。