The aligned Large Language Models (LLMs) are powerful language understanding and decision-making tools that are created through extensive alignment with human feedback. However, these large models remain susceptible to jailbreak attacks, where adversaries manipulate prompts to elicit malicious outputs that should not be given by aligned LLMs. Investigating jailbreak prompts can lead us to delve into the limitations of LLMs and further guide us to secure them. Unfortunately, existing jailbreak techniques suffer from either (1) scalability issues, where attacks heavily rely on manual crafting of prompts, or (2) stealthiness problems, as attacks depend on token-based algorithms to generate prompts that are often semantically meaningless, making them susceptible to detection through basic perplexity testing. In light of these challenges, we intend to answer this question: Can we develop an approach that can automatically generate stealthy jailbreak prompts? In this paper, we introduce AutoDAN, a novel jailbreak attack against aligned LLMs. AutoDAN can automatically generate stealthy jailbreak prompts by the carefully designed hierarchical genetic algorithm. Extensive evaluations demonstrate that AutoDAN not only automates the process while preserving semantic meaningfulness, but also demonstrates superior attack strength in cross-model transferability, and cross-sample universality compared with the baseline. Moreover, we also compare AutoDAN with perplexity-based defense methods and show that AutoDAN can bypass them effectively.
AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs
University of Wisconsin–Madison(威斯康星大学麦迪逊分校);NVIDIA(NVIDIA公司);Cornell University(康奈尔大学);Washington University, St. Louis(圣路易斯华盛顿大学);University of Michigan, Ann Arbor(安娜堡大学);The Ohio State University(俄亥俄州立大学);UIUC(伊利诺伊大学厄巴纳-香槟分校)
Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring
Harbin Institute of Technology(哈尔滨工业大学);Central South University(中南大学);East China Normal University(华东师范大学);Beihang University(北京航空航天大学);LibrAI;MBZUAI;Tsinghua University(清华大学)
ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models
iFLYTEK Security Laboratory(iFLYTEK安全实验室);Anhui SparkShield Intelligent Technology Co., Ltd.(安徽SparkShield智能科技有限公司);Peking University(北京大学);School of Automation, University of Electronic Science and Technology of China(电子科技大学自动化学院);Northwest University(西北大学);University of New South Wales(新南威尔士大学);National Computer Network Emergency Response Technical Team/Coordination Center of China(中国国家计算机网络应急技术处置协调中心)
The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
Department of Computer Science, University of Texas at El Paso(德克萨斯大学埃尔帕索分校计算机科学系);School of Computing, Southern Illinois University Carbondale(南方伊利诺伊大学卡罗尔丹分校计算学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors
Zhejiang University(浙江大学);Palo Alto Networks(帕洛阿尔托网络);Ningbo Global Innovation Center, Zhejiang University(宁波全球创新中心,浙江大学);OPPO Research Institute(OPPO研究院);Qilu University of Technology(齐鲁大学)
Engineering Programme, NUS Graduate School, National University of Singapore(国立新加坡大学整合科学与工程计划);Institute of Data Science (IDS), National University of Singapore(国立新加坡大学数据科学研究所);Department of Computer Science, School of Computing, National University of Singapore(国立新加坡大学计算机科学系)
SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models
Tsinghua University(清华大学);Shanghai Jiao Tong University(上海交通大学);Kuaishou Technology(快手科技);School of Computer Science and Technology, Beihang University(北航计算机科学与技术学院);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Chongqing University(重庆大学);Wuhan University(武汉大学)
Intention Analysis Makes LLMs A Good Jailbreak Defender
School of Computer Science, Wuhan University(武汉大学计算机学院);The University of Sydney(悉尼大学);College of Computing and Data Science at Nanyang Technological University(南洋理工大学计算与数据科学学院)