注意力滑移：对大语言模型中越狱攻击与防御的机制性理解
Attention Slipping: A Mechanistic Understanding of Jailbreak Attacks and Defenses in LLMs
AI总结：
本文揭示越狱攻击中的普遍现象“注意力滑移”，即模型逐渐减少对不安全请求的注意力分配，并提出基于温度缩放的注意力锐化防御方法，在四个主流大语言模型上有效抵御多种越狱攻击且不增加计算开销。
摘要：
随着大语言模型（LLMs）日益融入社会与技术，确保其安全性变得至关重要。越狱攻击利用漏洞绕过安全护栏，构成重大威胁。然而，使这些攻击得以实现的机制尚未被充分理解。本文揭示了一种在越狱攻击过程中出现的普遍现象：注意力滑移（Attention Slipping）。在这一现象中，模型在攻击过程中逐渐减少对用户查询中不安全请求所分配的注意力，最终导致越狱。我们表明，注意力滑移在多种越狱方法中表现一致，包括基于梯度的令牌替换、提示级模板优化以及上下文学习。此外，我们评估了两种基于查询扰动的防御方法，即Token Highlighter和SmoothLLM，发现它们间接缓解了注意力滑移，其有效性与所实现的缓解程度呈正相关。受此发现启发，我们提出了注意力锐化（Attention Sharpening），这是一种新的防御方法，通过使用温度缩放来锐化注意力分数分布，直接对抗注意力滑移。在四个领先的大语言模型（Gemma2-9B-It、Llama3.1-8B-It、Qwen2.5-7B-It、Mistral-7B-It v0.2）上的实验表明，我们的方法能够有效抵御各种越狱攻击，同时在AlpacaEval上保持良性任务的性能。重要的是，注意力锐化不引入额外的计算或内存开销，使其成为实际部署中高效且实用的解决方案。
英文摘要：
As large language models (LLMs) become more integral to society and technology, ensuring their safety becomes essential. Jailbreak attacks exploit vulnerabilities to bypass safety guardrails, posing a significant threat. However, the mechanisms enabling these attacks are not well understood. In this paper, we reveal a universal phenomenon that occurs during jailbreak attacks: Attention Slipping. During this phenomenon, the model gradually reduces the attention it allocates to unsafe requests in a user query during the attack process, ultimately causing a jailbreak. We show Attention Slipping is consistent across various jailbreak methods, including gradient-based token replacement, prompt-level template refinement, and in-context learning. Additionally, we evaluate two defenses based on query perturbation, Token Highlighter and SmoothLLM, and find they indirectly mitigate Attention Slipping, with their effectiveness positively correlated with the degree of mitigation achieved. Inspired by this finding, we propose Attention Sharpening, a new defense that directly counters Attention Slipping by sharpening the attention score distribution using temperature scaling. Experiments on four leading LLMs (Gemma2-9B-It, Llama3.1-8B-It, Qwen2.5-7B-It, Mistral-7B-It v0.2) show that our method effectively resists various jailbreak attacks while maintaining performance on benign tasks on AlpacaEval. Importantly, Attention Sharpening introduces no additional computational or memory overhead, making it an efficient and practical solution for real-world deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
