SCANS：通过安全意识激活引导缓解大语言模型的过度安全
SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering
AI总结：
提出SCANS方法，通过提取拒绝引导向量并锚定安全关键层、跟踪隐藏状态转换来引导模型行为，在XSTest和OKTest上取得最先进性能，平衡过度安全与防御能力。
摘要：
安全对齐对于大语言模型（LLMs）防御恶意指令的威胁至关重要。然而，近期研究揭示，经过安全对齐的LLMs因过度安全问题而倾向于拒绝良性查询，限制了其有用性。本文提出一种安全意识激活引导（SCANS）方法，以缓解对齐LLMs中的过度安全担忧。首先，SCANS在激活空间中提取拒绝引导向量，并利用词汇投影来锚定影响模型拒绝行为的特定安全关键层。其次，通过跟踪隐藏状态转换，SCANS识别引导方向并相应引导模型行为，在过度安全与充分安全之间取得平衡。实验表明，SCANS在XSTest和OKTest基准上取得了新的最先进性能，同时不损害其对有害查询的防御能力，并保持模型能力几乎不变。
英文摘要：
Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions. However, recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue, limiting their helpfulness. In this paper, we propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns in aligned LLMs. First, SCANS extracts the refusal steering vectors within the activation space and utilizes vocabulary projection to anchor some specific safety-critical layers which influence model refusal behavior. Second, by tracking the hidden state transition, SCANS identifies the steering direction and steers the model behavior accordingly, achieving a balance between exaggerated safety and adequate safety. Experiments show that SCANS achieves new state-of-the-art performance on XSTest and OKTest benchmarks, without impairing their defense capability against harmful queries and maintaining almost unchanged model capability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
