Jailbreaking in Large Language Models (LLMs) is a major security concern as it can deceive LLMs to generate harmful text. Yet, there is still insufficient understanding of how jailbreaking works, which makes it hard to develop effective defense strategies. We aim to shed more light into this issue: we conduct a detailed large-scale analysis of seven different jailbreak methods and find that these disagreements stem from insufficient observation samples. In particular, we introduce \textit{safety boundary}, and we find that jailbreaks shift harmful activations outside that safety boundary, where LLMs are less sensitive to harmful information. We also find that the low and the middle layers are critical in such shifts, while deeper layers have less impact. Leveraging on these insights, we propose a novel defense called \textbf{Activation Boundary Defense} (ABD), which adaptively constrains the activations within the safety boundary. We further use Bayesian optimization to selectively apply the defense method to the low and the middle layers. Our experiments on several benchmarks show that ABD achieves an average DSR of over 98\% against various forms of jailbreak attacks, with less than 2\% impact on the model's general capabilities.
Latent-space adversarial training with post-aware calibration for defending large language models against jailbreak attacks
Lab of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China（教育人工智能实验室，东华大学，上海200062，中国）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China（教育人工智能研究所，东华大学，上海200062，中国）；School of Computer Science and Technology, East China Normal University, Shanghai 200062, China（计算机科学与技术学院，东华大学，上海200062，中国）
Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking
University of Southern California（南加州大学）；University of Pennsylvania（宾夕法尼亚大学）；University of Wisconsin Madison（威斯康星大学麦迪逊分校）；University of California, Davis（加利福尼亚大学戴维斯分校）
Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
School of EECS, Peking University（电子工程系，北京大学）；School of Mathematical Sciences, Peking University（数学系，北京大学）；State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（通用人工智能国家重点实验室，智能科学与技术学校，北京大学）；Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）
Strategic Deflection: Defending LLMs from Logit Manipulation
International Artificial Intelligence Center of Morocco（摩洛哥国际人工智能中心）；Mohammed VI Polytechnic University（摩洛哥穆莱·伊斯梅尔理工学院）；AgriEdge；Sorbonne University, LIP6 - UMR 7606 CNRS（索邦大学，LIP6 - UMR 7606 CNRS）
Defending Against Malicious Finetuning by Scaling Train-time Adversarial Attacks
Xiongan AI Institute（雄安人工智能研究院）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Shanghai Qi Zhi Institute（上海期智研究院）
X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usability
School of Electronic Information and Electric Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Adversarial Tuning: Defending Against Jailbreak Attacks for LLMs
AI Thrust（AI 部）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Guangzhou HKUST Fok Ying Tung Research Institute（广州香港科技大学霍英东研究院）；The Hong Kong University of Science and Technology（香港科技大学）
ArtPerception: ASCII Art-based Jailbreak on LLMs with Recognition Pre-test
Department of Electrical Engineering, National Taiwan University（国立台湾大学电子工程系）；GARMIN (ASIA) CORPORATION（GARMIN（亚洲）公司）；Institute of Artificial Intelligence Innovation, National Yang Ming Chiao Tung University（国家阳明交通大学人工智能创新研究所）；Department of Information Management, National Dong Hwa University（国立东吴大学资讯管理系）
Advancing the Robustness of Large Language Models through Self-Denoised Smoothing
UC Santa Barbara（加州大学圣巴巴拉分校）；Max Planck Institute for Intelligent Systems, Tübingen（马克斯·普朗克智能系统研究所（蒂宾根））；The Hong Kong Polytechnic University（香港理工大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；Cisco Research（思科研究院）；Michigan State University（密歇根州立大学）
Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
Georgia Institute of Technology（佐治亚理工学院）；University of Bristol（布里斯托大学）；University of Maryland（马里兰大学）；University College London（伦敦大学学院）；MATS；Astra；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）
A Theoretical Understanding of Self-Correction through In-context Alignment
School of EECS, Peking University（北京大学电子工程与计算机科学学院）；School of Mathematical Sciences, Peking University（北京大学数学科学学院）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；TU Munich（慕尼黑工业大学）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）
