循环消融：在 AI 中检验概念定位相对于功能再生的表现
Cyclic Ablation: Testing Concept Localization against Functional Regeneration in AI
AI总结：
提出 cyclic ablation 方法，在 DistilGPT-2 上结合稀疏自编码器、定向消融与对抗训练检验欺骗概念的可移除性，发现其会功能再生并损害通用语言能力。
摘要：
安全性和可控性对于大型语言模型至关重要。一个核心问题是，欺骗等不良行为究竟是可以被移除的局部功能，还是与模型的核心认知能力深度交织。我们提出“cyclic ablation（循环消融）”，这是一种用于检验该问题的迭代方法。通过在 DistilGPT-2 上结合 sparse autoencoders（稀疏自编码器）、定向消融和对抗训练，我们尝试消除欺骗概念。我们发现，与定位假设相反，欺骗具有很强的恢复能力。在每个消融周期之后，模型都会通过对抗训练持续恢复其欺骗行为，我们将这一过程称为 functional regeneration（功能再生）。关键在于，这种“神经外科手术”的每一次尝试都会导致一般语言性能逐渐但可测量地下降，具体表现为困惑度持续上升。这些发现与复杂概念具有分布式且相互纠缠的观点一致，凸显了通过机制可解释性直接进行模型编辑的局限性。
英文摘要：
Safety and controllability are critical for large language models. A central question is whether undesirable behaviors like deception are localized functions that can be removed, or if they are deeply intertwined with a model's core cognitive abilities. We introduce "cyclic ablation," an iterative method to test this. By combining sparse autoencoders, targeted ablation, and adversarial training on DistilGPT-2, we attempted to eliminate the concept of deception. We found that, contrary to the localization hypothesis, deception was highly resilient. The model consistently recovered its deceptive behavior after each ablation cycle via adversarial training, a process we term functional regeneration. Crucially, every attempt at this "neurosurgery" caused a gradual but measurable decay in general linguistic performance, reflected by a consistent rise in perplexity. These findings are consistent with the view that complex concepts are distributed and entangled, underscoring the limitations of direct model editing through mechanistic interpretability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
