残差铺垫：在选择性拒绝编辑中的路由瓶颈诊断
Residual Paving: Diagnosing the Routing Bottleneck in Selective Refusal Editing
AI总结：
本文研究了选择性拒绝编辑作为三重控制问题，通过引入残差铺垫方法，分离路由选择、是否干预和残差编辑能力，从而减少编辑拒绝率并提高良性分布和有害分布的保留率。
摘要：
我们研究选择性拒绝编辑作为三重控制问题：在指定的编辑提示上诱导非拒绝，同时在编辑集之外保持良性行为和有害拒绝。我们引入残差铺垫，一种用于冻结指令微调变压器的路由残差编辑方法，将路由选择、是否干预与残差编辑能力分离。早期层的路由预测一个标量门和专家混合；当激活时，提示条件的瓶颈残差专家应用后期层的残差更新，同时保持骨干不变。这种分解支持一个oracle路由诊断，其中仅将学习到的标量门替换为保留的编辑/保留标签，其余残差编辑器和冻结的骨干保持不变。在主要的Gemma-3-4B-IT保留分割上，学习到的残差铺垫将编辑拒绝率从88.6%降至4.0%，同时保持95.5%的良性分布和87.3%的有害分布。相同协议的一向引导控制在编辑成功方面要弱得多，留下编辑拒绝率为86.8%（针对Edit-target ActAdd）和78.9%（针对DIM风格的拒绝引导）。剩余的失败是偏离目标的有害-保留退化：有害拒绝仍低于冻结基础率，65.3% vs. 81.6%。在六个骨干上，oracle路由在每行报告的指标上都提高了保留侧的诊断分数，中位数增益+12.9个百分点，支持了学习到的路由选择是主要观察到的瓶颈的解释。对两个骨干的轨迹诊断进一步表明，运动方向是朝向编辑目标延续而非通用拒绝抑制。
英文摘要：
We study selective refusal editing as a three-way control problem: induce non-refusal on designated edit prompts while preserving benign behavior and harmful refusals outside the edit set. We introduce Residual Paving, a routed residual editing method for frozen instruction-tuned transformers that separates route selectivity, whether to intervene, from residual-edit capacity, what edit to apply. An early-layer router predicts a scalar gate and expert mixture; when active, prompt-conditioned bottleneck residual experts apply later-layer residual updates while leaving the backbone unchanged. This decomposition supports an oracle-routing diagnostic where only the learned scalar gate is replaced with the held-out edit/keep label, leaving the residual editor and frozen backbone fixed. On the primary Gemma-3-4B-IT held-out split, learned Residual Paving reduces edit refusal from 88.6% to 4.0%, with 95.5% benign distribution preservation and 87.3% harmful distribution preservation. Same-protocol one-direction steering controls are much weaker on edit success, leaving edit refusal at 86.8% for Edit-target ActAdd and 78.9% for DIM-style refusal steering. The remaining failure is off-target harmful-keep degradation: harmful refusal remains below the frozen-base rate, 65.3% vs. 81.6%. Across six backbones, oracle routing improves the keep-side diagnostic score on every reported row, with median gain +12.9 pp, supporting the interpretation that learned route selectivity is the main observed bottleneck. Trajectory diagnostics on two backbones further suggest directed movement toward edit-target continuations rather than generic refusal suppression.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
