通过后Transformer适配器纠正语言模型中的压制对数概率
Correcting Suppressed Log-Probabilities in Language Models with Post-Transformer Adapters
AI总结：
研究通过后Transformer适配器纠正语言模型在敏感话题上的压制对数概率，展示适配器在不同规模模型上的泛化能力及生成一致性改进。
摘要：
对齐调节的语言模型经常在政治敏感话题上压制事实对数概率，尽管其隐藏表示仍保留知识。我们展示了一个786000参数（约基础模型的0.02%）的后Transformer适配器，在冻结隐藏状态上训练，能纠正Qwen3-4B、8B和14B模型上31个意识形态区分事实的压制。适配器记忆所有15个训练事实，并在5个随机分割中泛化至11-39%的16个保留事实，通过锚定训练实现零知识回归。门控（SwiGLU）和非门控（线性瓶颈）适配器表现相当；两者均不一致地优于对方（所有规模Fisher精确p>0.09）。在指令模型中，适配器纠正对数概率排名。当在生成过程中应用所有token位置时，适配器产生不连贯的输出；然而，仅在当前预测位置（最后位置-only）应用时，适配器产生连贯、较少受阻的文本。一个在token投影后操作的logit空间适配器在任何应用模式下都无法产生连贯的生成，表明隐藏状态干预是生成纠正的正确层级。之前未记录的Apple MLX中的静默梯度错误解释了早期迭代中所有零结果：标准模式nn.value_and_grad(model, fn)(model.parameters())返回零梯度无错误；正确模式nn.value_and_grad(model, fn)(model, data)解决此问题。我们提供了一个最小可复现方案，并讨论了MLX中其他适配器研究的含义。
英文摘要：
Alignment-tuned language models frequently suppress factual log-probabilities on politically sensitive topics despite retaining the knowledge in their hidden representations. We show that a 786K-parameter (approximately 0.02% of the base model) post-transformer adapter, trained on frozen hidden states, corrects this suppression on 31 ideology-discriminating facts across Qwen3-4B, 8B, and 14B. The adapter memorizes all 15 training facts and generalizes to 11--39% of 16 held-out facts across 5 random splits per scale, with zero knowledge regressions via anchored training. Both gated (SwiGLU) and ungated (linear bottleneck) adapters achieve comparable results; neither consistently outperforms the other (Fisher exact p > 0.09 at all scales). On instruct models, the adapter corrects log-probability rankings. When applied at all token positions during generation, the adapter produces incoherent output; however, when applied only at the current prediction position (last-position-only), the adapter produces coherent, less censored text. A logit-space adapter operating after token projection fails to produce coherent generation at any application mode, suggesting hidden-state intervention is the correct level for generation correction. A previously undocumented silent gradient bug in Apple MLX explains all null results in earlier iterations of this work: the standard pattern nn.value_and_grad(model, fn)(model.parameters()) returns zero gradients without error; the correct pattern nn.value_and_grad(model, fn)(model, data) resolves this. We provide a minimal reproduction and discuss implications for other adapter research using MLX.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
