单个神经元足以绕过大型语言模型的安全对齐
A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models
AI总结：
研究通过针对单个神经元展示安全对齐的两种失效方向，证明在不训练或提示工程的情况下，通过抑制或放大特定神经元可绕过安全对齐。
摘要：
语言模型中的安全对齐通过两种机制不同的系统运作：拒绝神经元决定是否表达有害知识，概念神经元编码有害知识本身。通过针对每个系统中的单个神经元，我们展示了两种失效方向——通过抑制绕过安全对齐，以及通过放大从无害提示诱导有害内容——在七个模型中，涵盖两个家族和17亿至700亿参数。我们的发现表明，安全对齐并非在模型权重中稳健分布，而是由个别神经元介导，每个神经元都足以因果地控制拒绝行为——抑制任何一个识别出的拒绝神经元可绕过安全对齐，适用于各种有害请求。
英文摘要：
Safety alignment in language models operates through two mechanistically distinct systems: refusal neurons that gate whether harmful knowledge is expressed, and concept neurons that encode the harmful knowledge itself. By targeting a single neuron in each system, we demonstrate both directions of failure -- bypassing safety on explicit harmful requests via suppression, and inducing harmful content from innocent prompts via amplification -- across seven models spanning two families and 1.7B to 70B parameters, without any training or prompt engineering. Our findings suggest that safety alignment is not robustly distributed across model weights but is mediated by individual neurons that are each causally sufficient to gate refusal behavior -- suppressing any one of the identified refusal neurons bypasses safety alignment across diverse harmful requests.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
