LARV：无数据层间自适应重标量 veneer 用于模型合并
LARV: Data-Free Layer-wise Adaptive Rescaling Veneer for Model Merging
AI总结：
LARV通过层间自适应重标量技术提升模型合并性能，无需训练或修改现有合并器，有效抑制浅层干扰并增强深层特征对齐。
摘要：
模型合并旨在将多个微调模型合并为一个多功能模型，而无需访问训练数据。现有的任务向量合并方法，如TIES、TSV-M和Iso-C/CTS，在聚合规则上有所不同，但都将近似均匀地对待所有层。这一假设忽略了大型视觉变换器中强的层间异质性，其中浅层对干扰敏感，而深层编码稳定的任务特定特征。我们引入了LARV，一种无需训练、无需数据、合并无关的层间自适应重标量veiner，它可以插入到任何任务向量合并中，并在聚合前为每个任务向量分配一个每层的缩放因子，并展示它一致地提高了多种合并规则。LARV通过简单的确定性计划，自适应地抑制浅层干扰并放大深层对齐，不需要重新训练或修改现有合并器。据我们所知，这是首次进行层感知缩放的任务向量合并工作。LARV计算简单的无数据层代理，并通过轻量级规则将其转换为缩放因子；我们研究了几个在同一个框架内的实例（例如，分层两/三层缩放，固定值或连续映射），并展示了分层选择提供了最佳的鲁棒性，而连续映射仍然是一个消融。LARV与基础合并器正交，并增加 negligible 成本。在FusionBench上使用视觉变换器时，LARV在8/14/20任务设置下一致提高了所有任务向量基线；例如，Iso-C + LARV在ViT-B/32上达到85.9%，在ViT-B/16上达到89.2%，在ViT-L/14上达到92.6%。层间分析和破坏测试进一步表明，LARV抑制了浅层干扰，同时适度放大了更深层、任务稳定的特征，将模型合并转化为一个稳健、层感知的程序，而不是一个统一的程序。
英文摘要：
Model merging aims to combine multiple fine-tuned models into a single multi-task model without access to training data. Existing task-vector merging methods such as TIES, TSV-M, and Iso-C/CTS differ in their aggregation rules but treat all layers nearly uniformly. This assumption overlooks the strong layer-wise heterogeneity in large vision transformers, where shallow layers are sensitive to interference while deeper layers encode stable task-specific features. We introduce LARV, a training-free, data-free, merger-agnostic Layer-wise Adaptive Rescaling Veneer that plugs into any task-vector merger and assigns a per-layer scale to each task vector before aggregation, and show it consistently boosts diverse merging rules. LARV adaptively suppresses shallow-layer interference and amplifies deeper-layer alignment using a simple deterministic schedule, requiring no retraining or modification to existing mergers. To our knowledge, this is the first work to perform layer-aware scaling for task-vector merging. LARV computes simple data-free layer proxies and turns them into scales through a lightweight rule; we study several instantiations within one framework (e.g., tiered two/three-level scaling with fixed values, or continuous mappings) and show that tiered choices offer the best robustness, while continuous mappings remain an ablation. LARV is orthogonal to the base merger and adds negligible cost. On FusionBench with Vision Transformers, LARV consistently improves all task-vector baselines across 8/14/20-task settings; for example, Iso-C + LARV reaches 85.9% on ViT-B/32, 89.2% on ViT-B/16, and 92.6% on ViT-L/14. Layerwise analysis and corruption tests further indicate that LARV suppresses shallow-layer interference while modestly amplifying deeper, task-stable features, turning model merging into a robust, layer-aware procedure rather than a uniform one.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
