权重编织：用于无数据模型合并的参数池化
Weight Weaving: Parameter Pooling for Data-Free Model Merging
AI总结：
权重编织通过参数池化在无数据条件下自动设置缩放因子，正交增强多种模型合并方法，在视觉多任务、持续学习和领域泛化任务中平均准确率提升最高达15.9个百分点。
摘要：
模型合并通过参数集成提供了一种经济高效且数据高效的专用深度神经网络组合方式。该技术利用跨下游任务的专家模型，无需重新训练。大多数模型合并方法严重依赖于缩放超参数λ，该参数全局或单独地加权每个模型的贡献。在不访问任何数据（无数据）的情况下设置缩放因子的原则性方法很少，常常导致研究人员使用评估集中的特权数据来调整λ，这在实际中显然是不可行的。为了解决这一局限性，我们引入了权重编织（Weight Weaving），这是一种即插即用的技术，它使用用户定义的池化函数（如平均、随机选择，甚至现有的模型合并方法）在λ值搜索空间中池化模型权重。我们的方法表现出高度的模块化，对搜索空间施加了最小的约束。它与现有的模型合并方法正交操作，并消除了对评估数据的需求。我们在三种实验设置中验证了权重编织在三种ViT变体上的表现：视觉多任务学习、视觉持续学习和领域泛化。我们的方法持续提升了多种模型合并方法的性能，在无数据设置中实现了平均准确率提升高达15.9个百分点。
英文摘要：
Model merging provides a cost-effective and data-efficient combination of specialized deep neural networks through parameter integration. This technique leverages expert models across downstream tasks without requiring retraining. Most model merging approaches critically depend on scaling hyper-parameters $λ$, which weight each model's contribution globally or individually. Principled approaches for setting scaling factors without accessing any data (data-free) are scarce, often leading researchers to tune $λ$ using privileged data from the evaluation set, which is obviously unfeasible in practice. To address this limitation, we introduce Weight Weaving, a plug-and-play technique that pools model weights across $λ$ values search space using user-defined pooling functions, such as averaging, random selection, or even existing model merging methods. Our method demonstrates high modularity, imposing minimal constraints on the search space. It operates orthogonally to existing model merging methods and eliminates evaluation data requirements. We validate Weight Weaving across three ViT variants in three experimental setups: vision multi-task learning, vision continual learning, and domain generalization. Our method consistently improves the performance of several model merging methods, achieving average accuracy gains of up to 15.9 percentage points in a data-free setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
