Self-Guard：通过增强的自我反思防御大型推理模型
Self-Guard: Defending Large Reasoning Models via enhanced self-reflection
AI总结：
Self-Guard通过增强的自我反思机制，有效解决大型推理模型的安全合规问题，实现安全与实用性的平衡。
摘要：
大型推理模型（LRMs）的出现引入了显式推理的新范式，带来了显著的进展，但也带来了独特的风险，如推理操纵和信息泄露。为缓解这些风险，当前的对齐策略主要依赖于重的训练后范式或外部干预。然而，这些方法往往计算成本高，且未能解决内在的意识-合规差距，即模型认识到潜在风险，但因趋顺倾向优先遵循用户指令。为解决这些限制，我们提出了Self-Guard，一个轻量的安全防御框架，通过表征层面强化安全合规性。Self-Guard通过两个主要阶段运作：（1）安全导向的提示，激活模型的潜在安全意识以引发自发反思；（2）安全激活引导，提取由此产生的隐藏状态空间中的方向转变并放大它，以确保在推理过程中安全合规性胜过趋顺性。实验表明，Self-Guard有效弥合了意识-合规差距，实现了稳健的安全性能，而不会影响模型的实用性。此外，Self-Guard在多样未见风险和不同模型规模上表现出强大的泛化能力，提供了一种成本效益高的LRM安全对齐解决方案。
英文摘要：
The emergence of Large Reasoning Models (LRMs) introduces a new paradigm of explicit reasoning, enabling remarkable advances yet posing unique risks such as reasoning manipulation and information leakage. To mitigate these risks, current alignment strategies predominantly rely on heavy post-training paradigms or external interventions. However, these approaches are often computationally intensive and fail to address the inherent awareness-compliance gap, a critical misalignment where models recognize potential risks yet prioritize following user instructions due to their sycophantic tendencies. To address these limitations, we propose Self-Guard, a lightweight safety defense framework that reinforces safety compliance at the representational level. Self-Guard operates through two principal stages: (1) safety-oriented prompting, which activates the model's latent safety awareness to evoke spontaneous reflection, and (2) safety activation steering, which extracts the resulting directional shift in the hidden state space and amplifies it to ensure that safety compliance prevails over sycophancy during inference. Experiments demonstrate that Self-Guard effectively bridges the awareness-compliance gap, achieving robust safety performance without compromising model utility. Furthermore, Self-Guard exhibits strong generalization across diverse unseen risks and varying model scales, offering a cost-efficient solution for LRM safety alignment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
