并非所有任务向量都需要相等的秩：模型合并中的能量比例分配

中文摘要 AI 辅助

模型合并旨在将源自同一预训练模型的多个微调模型组合成一个多任务模型，而无需额外的联合训练。最近的谱合并方法通过利用任务特定更新的低秩结构，改进了简单的权重平均方法，但它们通常为每个任务分配相同的秩容量。这种均匀分配忽略了任务向量可能具有异质的谱复杂度，导致共享合并空间的使用不够优化。在本文中，我们提出了谱能量比例秩分配（SERA），这是一种简单的任务自适应策略，根据每个任务向量的奇异值能量结构来分配秩。通过为复杂或孤立任务分配更丰富的谱容量，并为紧凑任务分配更少的方向，SERA将基于SVD的模型合并从均匀容量合并扩展到任务相关的容量分配。在标准视觉模型合并协议下的实验表明，SERA在保持与现有谱合并方法相同的总秩预算的同时，提高了多任务合并性能。进一步的分析表明，任务级别的谱集中度与自适应秩分配对每个任务的影响密切相关，为SERA何时以及为何有效提供了见解。

英文摘要

Model merging aims to combine multiple fine-tuned models derived from a common pretrained model into a single multi-task model without additional joint training. Recent spectral merging methods improve over simple weight averaging by exploiting low-rank structures of task-specific updates, but they commonly assign the same rank capacity to every task. This uniform allocation ignores that task vectors can have heterogeneous spectral complexity, causing the shared merging space to be used suboptimally. In this paper, we propose Spectral Energy-proportional Rank Allocation (SERA), a simple task-adaptive strategy that allocates ranks according to the singular-value energy structure of each task vector. By assigning richer spectral capacity to complex or isolated tasks and fewer directions to compact tasks, SERA extends SVD-based model merging from uniform-capacity merging to task-dependent capacity allocation. Experiments under standard vision model merging protocols show that SERA improves multi-task merging performance while preserving the same total rank budget as existing spectral merging methods. Further analysis demonstrates that task-level spectral concentration is closely related to the per-task effect of adaptive rank allocation, providing insight into when and why SERA is effective.

发表机构

• LG CNS
• ETRI（韩国电子通信研究院）
• UST（韩国科学技术联合大学院大学）

机构由 AI 辅助整理，请以论文原文为准。