arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.18756cs.LGcs.AIcs.CLcs.CR
收藏

迈向稀疏自编码器鲁棒性的理解

Towards Understanding the Robustness of Sparse Autoencoders

University of Virginia(弗吉尼亚大学);Independent Researcher(独立研究者)

Ahson Saiyed, Sabrina Sadiekh, Chirag Agarwal

AI总结:

研究通过在推理阶段集成预训练的稀疏自编码器,发现其能显著降低对抗攻击成功率,并揭示稀疏性与攻击成功率之间的剂量-反应关系,以及层间防御与性能的权衡。

摘要:

大型语言模型(LLMs)仍易受基于优化的 jailbreak 攻击,这些攻击利用内部梯度结构。尽管稀疏自编码器(SAEs)广泛用于可解释性,但其鲁棒性影响仍待探索。本文研究了在推理时将预训练 SAEs 集成到 transformer 剩余流中,不修改模型权重或阻断梯度。在四个模型家族(Gemma、LLaMA、Mistral、Qwen)和两个强白盒攻击(GCG、BEAST)及三个黑盒基准测试中,SAE 增强模型相对于未防护基线的 jailbreak 成功率降低了 5 倍,并减少跨模型攻击转移性。参数消融揭示(i)L0 稀疏性与攻击成功率呈单调剂量-反应关系,(ii)层依赖的防御-效用权衡,其中中间层在鲁棒性和干净性能之间平衡。这些发现与表征瓶颈假说一致:稀疏投影重塑了 jailbreak 攻击所利用的优化几何结构。

英文摘要:

Large Language Models (LLMs) remain vulnerable to optimization-based jailbreak attacks that exploit internal gradient structure. While Sparse Autoencoders (SAEs) are widely used for interpretability, their robustness implications remain underexplored. We present a study of integrating pretrained SAEs into transformer residual streams at inference time, without modifying model weights or blocking gradients. Across four model families (Gemma, LLaMA, Mistral, Qwen) and two strong white-box attacks (GCG, BEAST) plus three black-box benchmarks, SAE-augmented models achieve up to a 5x reduction in jailbreak success rate relative to the undefended baseline and reduce cross-model attack transferability. Parametric ablations reveal (i) a monotonic dose-response relationship between L0 sparsity and attack success rate, and (ii) a layer-dependent defense-utility tradeoff, where intermediate layers balance robustness and clean performance. These findings are consistent with a representational bottleneck hypothesis: sparse projection reshapes the optimization geometry exploited by jailbreak attacks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑