SAE干预不可靠:干预后抑制行为的恢复
SAE Interventions are Unreliable: Post-Intervention Recovery of Suppressed Behavior
AI总结:
研究发现稀疏自编码器(SAE)特征干预虽能抑制行为,但存在可恢复的失败模式,通过优化残差扰动可恢复原始行为,揭示特征级控制与行为完整性之间的差距。
摘要:
稀疏自编码器(SAE)将残差流激活分解为可解释特征。最近的潜在空间防御越来越依赖这些分解,假设识别出的“不安全”SAE特征可作为监控和干预的可操作手柄。在这种范式下,固定特定有害特征预期能可靠地防止模型不当行为。然而,我们表明这种成功可能隐藏一种可恢复的失败模式:固定可能阻止行为的一条可见路径,但并未消除行为本身。我们将这种脆弱性形式化为干预后恢复,这是一个受约束的残差空间优化问题。从干预后的残差状态开始,我们优化残差扰动以恢复干预前的行为,同时保持目标SAE特征的干预后值。即使在干预在优化和生成过程中保持活跃的强威胁模型下,恢复仍然可能。为了排除恢复仅仅是撤销干预的可能性,我们使用编码器正交更新进行单层干预,并在跨层设置中使用相应的特征图雅可比矩阵。在TPP、遗忘、IOI和拒绝引导实验中,这种压力测试揭示了尽管特征级干预成功,行为仍可恢复。特别是在安全关键的拒绝引导设置中,我们在有效样本上实现了95.8%的恢复率,同时将防御特征的相对漂移保持在0.131,远低于基于后缀的基线。恢复路径归因分析进一步将这种恢复定位到SAE重建残差,即SAE未解释的组件。这些结果暴露了特征级控制与行为完整性之间的差距:SAE特征可以支持因果干预,但控制它们并不能保证对底层行为的控制。
英文摘要:
Sparse Autoencoders (SAEs) decompose residual-stream activations into interpretable features. Recent latent-space defenses increasingly rely on these decompositions, assuming that identified "unsafe" SAE features serve as actionable handles for monitoring and intervention. In this paradigm, clamping a specific harmful feature is expected to reliably prevent model misbehavior. However, we show that this success may hide a recoverable failure mode: the clamp may block one visible route to a behavior without eliminating the behavior itself. We formulate this vulnerability as post-intervention recovery, a constrained residual-space optimization problem. Starting from the post-intervention residual state, we optimize residual perturbations to recover the pre-intervention behavior while preserving the post-intervention values of the targeted SAE features. Even under a strong threat model where the intervention remains active throughout optimization and generation, recovery remains possible. To rule out that recovery simply undoes the intervention, we use encoder-orthogonal updates for single-layer interventions and the corresponding feature-map Jacobian in the cross-layer setting. Across TPP, unlearning, IOI, and refusal steering experiments, this stress test reveals recoverable behavior despite successful feature-level intervention. Especially in the safety-critical refusal-steering setting, we achieve a 95.8% recovery rate on valid samples while keeping defended-feature relative drift to 0.131, substantially below suffix-based baselines. A recovery-path attribution analysis further localizes this recovery to the SAE reconstruction residual, the component left unexplained by the SAE. These results expose a gap between feature-level control and behavioral completeness: SAE features can support causal intervention, but controlling them does not guarantee control over the underlying behavior.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。