保持独特，保持高效：在多任务合并中保留模型个性
Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging
AI总结：
本文提出DTS框架，通过奇异值分解、阈值策略和缩放机制，在多任务合并中提升存储效率，实验表明其在存储占用和泛化性能上优于现有方法。
摘要：
模型合并已成为一种有前景的范式，用于在不额外训练的情况下实现多任务能力。然而，传统基础合并方法由于参数冲突常导致性能下降，即使应用于相似任务。尽管最近的个性化合并框架成功保留了任务特定信息以维持性能，但通常会带来存储开销。在本文中，我们提出Decomposition, Thresholding, and Scaling (DTS)，一种基于近似的个性化合并框架，旨在提升任务特定的存储效率。DTS首先对任务特定信息应用奇异值分解，并仅保留少量奇异值和向量。然后引入一种新的阈值策略，将奇异向量元素分成组并为每个组分配缩放因子。为了使模型能够泛化到未见过的任务，我们进一步扩展DTS，通过基于任务特征语义相似性的方式，在无数据的情况下融合任务特定信息。大量实验表明，DTS在存储占用方面仅需每个任务1%的额外存储，同时在性能上始终优于最先进的基线方法。此外，针对未见过的任务的实验表明，DTS变体实现了显著更好的泛化性能。我们的代码可在https://github.com/krumpguo/DTS上获得。
英文摘要：
Model merging has emerged as a promising paradigm for enabling multi-task capabilities without additional training. However, traditional basic merging methods often experience performance degradation due to parameter conflicts, even when applied to similar tasks. While recent personalized merging frameworks successfully preserve task-specific information to maintain performance, they typically incur storage overhead. In this paper, we propose Decomposition, Thresholding, and Scaling (DTS), an approximation-based personalized merging framework that pushes task-specific storage efficiency. DTS first applies singular value decomposition to the task-specific information and retains only a small subset of singular values and vectors. It then introduces a novel thresholding strategy that partitions singular vector elements into groups and assigns a scaling factor to each group. To enable generalization to unseen tasks, we further extend DTS with a variant that fuses task-specific information in a data-free manner based on the semantic similarity of task characteristics. Extensive experiments demonstrate that DTS consistently outperforms state-of-the-art baselines while requiring only 1\% extra storage per task. Furthermore, experiments on unseen tasks show that the DTS variant achieves significantly better generalization performance. Our code is available at https://github.com/krumpguo/DTS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
