Large Language Models (LLMs), such as ChatGPT, encounter `jailbreak' challenges, wherein safeguards are circumvented to generate ethically harmful prompts. This study introduces a straightforward black-box method for efficiently crafting jailbreak prompts, addressing the significant complexity and computational costs associated with conventional methods. Our technique iteratively transforms harmful prompts into benign expressions directly utilizing the target LLM, predicated on the hypothesis that LLMs can autonomously generate expressions that evade safeguards. Through experiments conducted with ChatGPT (GPT-3.5 and GPT-4) and Gemini-Pro, our method consistently achieved an attack success rate exceeding 80% within an average of five iterations for forbidden questions and proved robust against model updates. The jailbreak prompts generated were not only naturally-worded and succinct but also challenging to defend against. These findings suggest that the creation of effective jailbreak prompts is less complex than previously believed, underscoring the heightened risk posed by black-box jailbreak attacks.
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
PRISM AI（PRISM人工智能公司）；Harmony Intelligence（和谐智能公司）；Leap Laboratories（leap实验室）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；ETH AI Center, ETH Zurich（苏黎世联邦理工学院人工智能中心）
PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximization
Alibaba Group（阿里巴巴集团）；Beijing Electronic Science and Technology Institute（北京电子科技研究所）；Nanjing University（南京大学）；Renmin University of China（中国人民大学）；Northeastern University（东北大学）；BraneMatrix AI；Nanyang Technological University（南洋理工大学）
Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring
Harbin Institute of Technology（哈尔滨工业大学）；Central South University（中南大学）；East China Normal University（华东师范大学）；Beihang University（北京航空航天大学）；LibrAI；MBZUAI；Tsinghua University（清华大学）
Audio Jailbreak Attacks: Exposing Vulnerabilities in SpeechGPT in a White-Box Framework
Department of Computer Science University of Missouri-Kansas City（计算机科学系 密苏里大学-康科特分校）；Department of Computer Science and Physics Rider University（计算机科学与物理系 Rider大学）；Department of Electrical and Computer Engineering University of Hawai’i at Mānoa（电气与计算机工程系 夏威夷大学马诺亚分校）
City University of Hong Kong（香港城市大学）；Rutgers University（罗格斯大学）；University of Science and Technology of China（中国科学技术大学）；Universiteit van Amsterdam（阿姆斯特丹大学）；Baidu Inc（百度公司）
Causal Front-Door Adjustment for Robust Jailbreak Attacks on LLMs
Institute of Software Chinese Academy of Sciences, Beijing, China（中国科学院软件研究所）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学）；Institute of Information Engineering Chinese Academy of Sciences, Beijing, China（中国科学院信息工程研究所）；Hong Kong University of Science and Technology, China, Hong Kong, China（香港科学与技术大学）
Speechmatics（Speechmatics公司）；MATS（机器学习对齐理论与实验项目（注：MATS全称为Machine Learning Alignment Theory and Experiments，为独立研究机构，此处保留缩写并补充说明））；UCL（伦敦大学学院）；Stanford University（斯坦福大学）；University of Oxford（牛津大学）；Tangentic（Tangentic公司）；Anthropic（Anthropic公司）
PIG: Privacy Jailbreak Attack on LLMs via Gradient-based Iterative In-Context Optimization
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络安全学院）；Cyberspace Institute of Advanced Technology, Guangzhou University（广州大学网络空间技术研究院）
Geneshift: Impact of different scenario shift on Jailbreaking LLM
National University of Singapore（新加坡国立大学）；NUS Graduate School（新加坡国立大学研究生院）；Integrative Sciences and Engineering Programme（整合科学与工程项目）；Institute of Data Science (IDS)（数据科学研究院）；School of Computing（计算机学院）
Obscure but Effective: Classical Chinese Jailbreak Prompt Optimization via Bio-Inspired Search
Nanyang Technological University（南洋理工大学）；BraneMatrix AI；Nanjing University of Science and Technology（南京理工大学）；Northeast University（东北大学）；Renmin University of China（中国人民大学）；Alibaba Group（阿里巴巴集团）；Beihang University（北航）；National University of Singapore（新加坡国立大学）；Zhejiang Lab（浙江实验室）
DualBreach: Efficient Dual-Jailbreaking via Target-Driven Initialization and Multi-Target Optimization
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China（区块链与数据安全国家重点实验室，浙江大学，杭州，中国）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China（杭州高新技术区（滨江）区块链与数据安全研究院，杭州，中国）
Activation-Guided Local Editing for Jailbreaking Attacks
Beihang University（北航）；Hangzhou Innovation Institute（杭州创新研究院）；HKUST（香港科技大学）；South China University of Technology（华南理工大学）；Nanyang Technological University（南洋理工大学）；Kunming University of Science and Technology（昆明理工大学）
Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Space
Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；College of AI, Tsinghua University（清华大学人工智能学院）；Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University（清华大学计算机科学与技术系）
Mitigating Many-shot Jailbreak Attacks with One Single Demonstration
Zhejiang University（浙江大学）；Nanyang Technological University（南洋理工大学）；Sun Yat-sen University（中山大学）；University of Science and Technology of China（中国科学技术大学）；Virginia Tech（弗吉尼亚理工大学）
MIST: Jailbreaking Black-box Large Language Models via Iterative Semantic Tuning
School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机科学与信息工程学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Cyber Security, Jinan University（暨南大学网络安全学院）
DoubtProbe: Black-Box Jailbreak Defense via Structural Verification and Semantic Auditing
East China Normal University（东华大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）
