HyperFix：面向任务向量合并的组合非线性修正方法
HyperFix: Combinatorial Nonlinear Correction for Task Vector Merging
浏览论文内容
中文总结 AI 辅助
HyperFix是一种轻量级超网络，可解决任务向量合并中跨子集重复调优及线性合并的局限，经小任务子集训练后泛化至更大子集，性能优于现有方法且调优成本更低。
中文摘要 AI 辅助
任务向量可实现无需联合再训练的模型合并。实际应用中，待合并的任务向量子集可能存在差异，但现有诸多方法针对特定子集采用标量调优，需跨子集重复调优，且将任务向量合并限制为线性重缩放。因此，我们将跨不同任务子集的合并问题形式化为组合修正问题，并提出HyperFix——一种轻量级超网络，可在权重空间中预测依赖子集的非线性修正。HyperFix仅在任务库的单元素、双元素及三元素子集上训练，即可泛化至更大子集，无需针对每个子集进行优化。我们的局部扰动分析界定了超出线性合并的残差修正，并启发从微小任务更新中学习该修正。在多种基准上开展的实验表明，HyperFix在降低调优成本的同时，性能优于现有任务向量合并方法。
英文摘要
Task vectors enable model merging without joint retraining. In practice, the subset of task vectors to be merged may vary, but many existing methods use scalar tuning for a particular subset, requiring repeated tuning across subsets and restricting task vector merging to linear rescaling. We therefore formulate merging across varying task subsets as a combinatorial correction problem and introduce HyperFix, a lightweight hypernetwork that predicts subset-conditioned nonlinear corrections in weight space. Trained once on singleton, pair, and triple subsets from a task bank, HyperFix generalizes to larger subsets without per-subset optimization. Our local perturbation analysis bounds the residual correction beyond linear merging and motivates learning it from small task updates. Experiments across diverse benchmarks show that HyperFix outperforms existing task vector merging methods while reducing tuning cost.
