发表机构
Sapienza University of Rome; Paradigma（罗马第一大学; Paradigma）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究探究模型合并对涌现能力的影响，发现合并保留双亲共有的涌现能力、无法创造超可加性能力，且会更快稀释单侧携带的能力，表明涌现行为不可组合。
AI 中文摘要
微调检查点和适配器如今充斥着公共仓库，而对这些工件最常见的操作是模型合并：对其权重进行算术运算，以低成本地组装能力。我们探究这一操作对涌现能力（工件所携带的、从未作为显式训练目标的行为）产生的影响。通过研究三个模型家族中的两个独立测试平台（激活预言机和涌现错位模型），我们发现答案有三方面。首先，合并会保留双亲都携带的涌现能力：合并两个错位检查点会在整个混合范围内保留其大部分广泛的错位。其次，合并无法创造出在双亲中具有超可加性的涌现能力：两个单任务预言机的任何加权合并都无法达到联合训练预言机的审计能力。第三，当只有一个双亲携带该能力时，合并对其的稀释速度快于伴随的训练能力：在大多数设置中，这一差距显著。简言之，工件的涌现行为不像其训练能力那样可组合。
英文摘要
Fine-tuned checkpoints and adapters now fill public repositories, and the most common operation applied to these artifacts is model merging: arithmetic on their weights that assembles capabilities cheaply. We ask what this operation does to emergent capabilities: behaviors an artifact carries that were never an explicit training target. Studying two independent testbeds (activation oracles and emergent-misaligned models) across three model families, we find that the answer is threefold. First, merging preserves an emergent capability that both parents carry: merging two misaligned checkpoints retains most of their broad misalignment across the whole mixing range. Second, merging cannot create an emergent capability that is superadditive in its parents: no weighted merge of two single-task oracles reaches the jointly-trained oracle's auditing ability. Third, when only one parent carries the capability, merging dilutes it faster than the trained capability that accompanies it: the gap is significant in most settings. In short, emergent behaviors of an artifact do not compose the way its trained capability does.
Commentsmain paper has 8 pages, 5 figures, and 4 tables
