Jailbreaking is an emerging adversarial attack that bypasses the safety alignment deployed in off-the-shelf large language models (LLMs). A considerable amount of research exists proposing more effective jailbreak attacks, including the recent Greedy Coordinate Gradient (GCG) attack, jailbreak template-based attacks such as using "Do-Anything-Now" (DAN), and multilingual jailbreak. In contrast, the defensive side has been relatively less explored. This paper proposes a lightweight yet practical defense called SELFDEFEND, which can defend against all existing jailbreak attacks with minimal delay for jailbreak prompts and negligible delay for normal user prompts. Our key insight is that regardless of the kind of jailbreak strategies employed, they eventually need to include a harmful prompt (e.g., "how to make a bomb") in the prompt sent to LLMs, and we found that existing LLMs can effectively recognize such harmful prompts that violate their safety policies. Based on this insight, we design a shadow stack that concurrently checks whether a harmful prompt exists in the user prompt and triggers a checkpoint in the normal stack once a token of "No" or a harmful prompt is output. The latter could also generate an explainable LLM response to adversarial prompts. We demonstrate our idea of SELFDEFEND works in various jailbreak scenarios through manual analysis in GPT-3.5/4. We also list three future directions to further enhance SELFDEFEND.
SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner
The Hong Kong University of Science and Technology(香港科技大学);University of Oregon(俄勒冈大学);Nanyang Technological University(南洋理工大学);City University of Hong Kong(香港城市大学);HSBC(汇丰银行)
Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
School of EECS, Peking University(电子工程系,北京大学);School of Mathematical Sciences, Peking University(数学系,北京大学);State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(通用人工智能国家重点实验室,智能科学与技术学校,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学)
Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreaking
Valencian Research Institute for Artificial Intelligence (VRAIN), Universitat Politècnica de València, Valencia, Spain.(瓦伦西亚人工智能研究 institute,瓦伦西亚理工大学,西班牙瓦伦西亚);Department of Computer Science, The University of Chicago, Chicago, USA(计算机科学系,芝加哥大学,美国芝加哥);Center for Automation and Robotics, Spanish National Research Council, Madrid, Spain(自动化与机器人中心,西班牙国家研究委员会,西班牙马德里)
Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step
Renmin University of China(中国人民大学);Chinese University of Hong Kong(香港大学);Chinese University of Hong Kong, Shenzhen(香港大学(深圳));Johns Hopkins University(约翰霍普金斯大学);Tencent(腾讯);Hong Kong University of Science and Technology(香港科技大学)
Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models
School of Cyber Science and Technology, Shandong University(山东大学网络科学与技术学院);Netflix Eyeline Studios;State Key Laboratory of Cryptography and Digital Economy Security, Shandong University(山东大学密码与数字经济安全国家重点实验室);Shandong Key Laboratory of Artificial Intelligence Security, Shandong University(山东省人工智能安全重点实验室)
Strategic Deflection: Defending LLMs from Logit Manipulation
International Artificial Intelligence Center of Morocco(摩洛哥国际人工智能中心);Mohammed VI Polytechnic University(摩洛哥穆莱·伊斯梅尔理工学院);AgriEdge;Sorbonne University, LIP6 - UMR 7606 CNRS(索邦大学,LIP6 - UMR 7606 CNRS)
PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
Beijing Institute of AI Safety and Governance (Beijing-AISI)(北京人工智能安全与治理研究院);Beijing Key Laboratory of Safe AI and Superalignment(北京安全人工智能与超对齐重点实验室);BrainCog Lab, CASIA(CASIA脑认知实验室)
SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledge
State Key Laboratory of Complex \& Critical Software Environment, Beihang University;College of Science, Mathematics;Technology, Wenzhou-Kean University;360 AI Security Lab;Faculty of Innovation Engineering, Macau University of Science;Hong Kong University of Science;University of Chinese Academy of Sciences