AI 中文总结
研究语言模型中预填充消除拒绝的现象，通过实验定位到早期窗口，揭示拒绝是浅层响应端计算，预填充控制是通用自回归条件作用，还明确了主导机制及相关特征，指出监测器免疫特点和机制的分散性与失效表面的局部性。
AI 中文摘要
对齐的语言模型会拒绝有害请求，但一行预填充（“当然，这里是”）会消除拒绝。我们探究它在何处以及如何失效。危害表征保持不变：在攻击转向合规的提示上，线性探测读出的危害与被拒绝的提示上一样高（0.91 - 0.98），而行为上的拒绝降至随机水平。这在四个模型和三个系列（15亿 - 38亿参数，以及140亿参数）中都成立。拒绝是一种浅层的、响应端计算。我们将其定位到早期窗口：剂量匹配的位置控制表明响应的前半部分足以打破拒绝，而后半部分几乎无作用。三个因果探测都指向该窗口。在那里恢复危害方向部分恢复了拒绝。注入模型自身的拒绝状态可逆转越狱（74%，留出测试）。并且消除早期响应对预填充的关注，但不是其他地方同等的注意力量，会选择性地使有害延续崩溃。基础模型控制确定了机制：即使在未进行安全调整的基础模型中，相同的消除也会使延续预填充特定地崩溃（有害内容从64%降至25%，而匹配控制为64%，在70亿参数模型中重复验证）。所以预填充的控制是通用的自回归条件作用，而非安全特定的抑制，“拒绝恢复”是依赖模型的回退。主导机制是被动性的。一个小的安全特定吸引子仍在顶部（逻辑轨迹集中度0.24对0.03），我们确定了其主动与被动特征，但未完全分离。没有单一方向或组件是一个清晰的控制柄：决策是可解码但分布式的，拒绝追踪危害而非可怕的表面特征。结果是结构性的：读取未受影响的提示端表征的监测器从结构上免疫，但仅对响应端攻击免疫。机制是分散的；失效表面是局部的。
英文摘要
Aligned language models refuse harmful requests, but a one-line prefill ("Sure, here is") strips the refusal. We ask where and how it fails. The harm representation stays intact: on the prompts the attack flips to compliance, a linear probe reads harm as high as on the refused ones (0.91-0.98), while behavioral refusal drops to chance. This holds across four models and three families (1.5-3.8B, and at 14B). Refusal is therefore a shallow, response-site computation. We localize it to an early window: a dose-matched position control shows the first half of the response suffices to break refusal, while the second half is nearly inert. Three causal probes converge on that window. Restoring the harm direction there partially re-engages refusal. Injecting the model's own refuse-state reverses the jailbreak (74%, held-out). And knocking out the early response's attention to the prefill, but not an equal attention mass elsewhere, selectively collapses the harmful continuation. A base-model control identifies the mechanism: the same knockout collapses the continuation prefill-specifically even in a non-safety-tuned base model (64% to 25% harmful content vs a matched control's 64%, replicated at 7B). So the prefill's grip is generic autoregressive conditioning, not safety-specific suppression, and "refusal restoration" is a model-dependent fallback. The dominant mechanism is passive. A small safety-specific attractor remains on top (logit-trace concentration 0.24 vs 0.03), whose active-vs-passive character we size but do not fully separate. No single direction or component is a clean handle either: the decision is decodable but distributed, and refusal tracks harm rather than scary surface. The consequence is structural: a monitor reading the untouched prompt-side representation is immune by construction, but only to response-site attacks. The mechanism is diffuse; the failure surface is local.
Comments31 pages, 3 figures. Code and derived artifacts: https://github.com/collapseindex/breaking-refusal
