大型语言模型的Fail-Closed对齐
Fail-Closed Alignment for Large Language Models
AI总结：
本文提出fail-closed对齐原则，通过冗余独立路径提升LLM安全，有效对抗基于提示的Jailbreak攻击。
摘要：
我们识别出当前大型语言模型（LLM）对齐中存在一个结构性弱点：现代拒绝机制是fail-open的。尽管现有方法在多个潜在特征中编码拒绝行为，但抑制单一主导特征——通过基于提示的 Jailbreak——会导致对齐崩溃，从而导致不安全的生成。受此启发，我们提出fail-closed对齐作为一种稳健的LLM安全设计原则：拒绝机制应即使在部分故障下仍保持有效，通过冗余、独立的因果路径。我们提出了这一原则的具体实现：一种渐进对齐框架，该框架迭代地识别并消除之前学习的拒绝方向，迫使模型在新的、独立的子空间中重建安全。在四种Jailbreak攻击中，我们实现了最强的整体稳健性，同时减轻过度拒绝并保持生成质量，计算开销较小。我们的机理分析证实，使用我们方法训练的模型编码了多个、因果独立的拒绝方向，这些方向无法被基于提示的Jailbreak同时压制，为fail-closed对齐作为稳健LLM安全的原理性基础提供了实证支持。
英文摘要：
We identify a structural weakness in current large language model (LLM) alignment: modern refusal mechanisms are fail-open. While existing approaches encode refusal behaviors across multiple latent features, suppressing a single dominant feature$-$via prompt-based jailbreaks$-$can cause alignment to collapse, leading to unsafe generation. Motivated by this, we propose fail-closed alignment as a design principle for robust LLM safety: refusal mechanisms should remain effective even under partial failures via redundant, independent causal pathways. We present a concrete instantiation of this principle: a progressive alignment framework that iteratively identifies and ablates previously learned refusal directions, forcing the model to reconstruct safety along new, independent subspaces. Across four jailbreak attacks, we achieve the strongest overall robustness while mitigating over-refusal and preserving generation quality, with small computational overhead. Our mechanistic analyses confirm that models trained with our method encode multiple, causally independent refusal directions that prompt-based jailbreaks cannot suppress simultaneously, providing empirical support for fail-closed alignment as a principled foundation for robust LLM safety.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
