语言模型中激活引导的内生抵抗
Endogenous Resistance to Activation Steering in Language Models
AI总结：
研究发现大型语言模型在任务不匹配的激活引导下能内生抵抗，通过显式重启恢复正确生成，并识别出相关稀疏自编码器潜在变量，可增强或削弱该抵抗。
摘要：
大型语言模型可以在生成过程中从任务不匹配的激活引导中恢复，产生显式的语言重启（例如，“等等，那不对”），并在引导扰动仍然活跃的情况下继续讨论主题。我们将此称为内生引导抵抗（ESR）。使用稀疏自编码器（SAE）潜在变量来引导模型激活，我们发现Llama-3.3-70B在\llamaseventyEsrRate\\%的情况下表现出显式ESR，而来自Llama-3和Gemma-2系列的较小模型则较少出现显式形式。两个对照实验将ESR分解为检测事件和持续抵抗组件，后者不能仅由最近的on-topic token条件化来完全解释。我们通过对比on-topic/off-topic搜索识别出\numOtdLatents{}个SAE潜在变量；将其零消融使多次尝试率降低\multiAttemptReductionPct\\%，随机潜在变量和保留提示对照支持特异性。ESR还可以通过元提示和基于合成自我纠正示例的微调来有意增强。ESR对安全性具有双重影响：它可能使模型对对抗性激活空间操纵更具抵抗力，但同样可能干扰有益的基于引导的干预，因为模型无法区分两者。代码可在\href{https://github.com/agencyenterprise/endogenous-steering-resistance}{github.com/agencyenterprise/endogenous-steering-resistance}获取。
英文摘要：
Large language models can recover mid-generation from task-misaligned activation steering, producing explicit verbal restarts (e.g., ``wait, that's not right'') and continuing on-topic even while the steering perturbation remains active. We term this Endogenous Steering Resistance (ESR). Using sparse autoencoder (SAE) latents to steer model activations, we find that Llama-3.3-70B exhibits explicit ESR at 3.8%, with smaller models from the Llama-3 and Gemma-2 families showing the explicit form less frequently. Two controls dissociate ESR into a detection event and a sustained-resistance component that conditioning on recent on-topic tokens does not fully explain. We identify 26 SAE latents through contrastive on-topic/off-topic search; zero-ablating them reduces the multi-attempt rate by 25%, with random-latent and held-out-prompt controls supporting specificity. ESR can also be deliberately enhanced through both meta-prompting and fine-tuning on synthetic self-correction examples. ESR has dual implications for safety: it could harden models against adversarial activation-space manipulation, but may equally interfere with beneficial steering-based interventions, since the model has no way to distinguish the two. Code is available at https://github.com/agencyenterprise/endogenous-steering-resistance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
