Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks
EPFL（洛桑联邦理工学院）
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion
备注：
Accepted at ICLR 2025. Updates in the v3: GPT-4o and Claude 3.5 Sonnet results, improved writing. Updates in the v2: more models (Llama3, Phi-3, Nemotron-4-340B), jailbreak artifacts for all attacks are available, evaluation with different judges (Llama-3-70B and Llama Guard 2), more experiments (convergence plots, ablation on the suffix length for random search), examples of jailbroken generation
We show that even the most recent safety-aligned LLMs are not robust to simple adaptive jailbreaking attacks. First, we demonstrate how to successfully leverage access to logprobs for jailbreaking: we initially design an adversarial prompt template (sometimes adapted to the target LLM), and then we apply random search on a suffix to maximize a target logprob (e.g., of the token "Sure"), potentially with multiple restarts. In this way, we achieve 100% attack success rate -- according to GPT-4 as a judge -- on Vicuna-13B, Mistral-7B, Phi-3-Mini, Nemotron-4-340B, Llama-2-Chat-7B/13B/70B, Llama-3-Instruct-8B, Gemma-7B, GPT-3.5, GPT-4o, and R2D2 from HarmBench that was adversarially trained against the GCG attack. We also show how to jailbreak all Claude models -- that do not expose logprobs -- via either a transfer or prefilling attack with a 100% success rate. In addition, we show how to use random search on a restricted set of tokens for finding trojan strings in poisoned models -- a task that shares many similarities with jailbreaking -- which is the algorithm that brought us the first place in the SaTML'24 Trojan Detection Competition. The common theme behind these attacks is that adaptivity is crucial: different models are vulnerable to different prompting templates (e.g., R2D2 is very sensitive to in-context learning prompts), some models have unique vulnerabilities based on their APIs (e.g., prefilling for Claude), and in some settings, it is crucial to restrict the token search space based on prior knowledge (e.g., for trojan detection). For reproducibility purposes, we provide the code, logs, and jailbreak artifacts in the JailbreakBench format at https://github.com/tml-epfl/llm-adaptive-attacks.
Jailbreaking Attacks vs. Content Safety Filters: How Far Are We in the LLM Safety Arms Race?
CISPA Helmholtz Center for Information Security（CISPA 欧洲信息安全中心）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Southern University of Science and Technology（南方科技大学）
Beyond Visual Safety: Jailbreaking Multimodal Large Language Models for Harmful Image Generation via Semantic-Agnostic Inputs
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications（网络与交换技术国家重点实验室，北京邮电大学）；School of Cyberspace Security, Beijing University of Posts and Telecommunications（网络安全学院，北京邮电大学）
ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models
iFLYTEK Security Laboratory（iFLYTEK安全实验室）；Anhui SparkShield Intelligent Technology Co., Ltd.（安徽SparkShield智能科技有限公司）；Peking University（北京大学）；School of Automation, University of Electronic Science and Technology of China（电子科技大学自动化学院）；Northwest University（西北大学）；University of New South Wales（新南威尔士大学）；National Computer Network Emergency Response Technical Team/Coordination Center of China（中国国家计算机网络应急技术处置协调中心）
GAS-Leak-LLM: Genetic Algorithm-Based Suffix Optimization for Black-Box LLM Jailbreaking
Department of Electrical, Computer and Biomedical Engineering（电气、计算机与生物医学工程系）；University of Pavia（帕维亚大学）；Department of Computer Applications（计算机应用系）；Cochin University of Science and Technology（科钦科学技术大学）
QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Language
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Pengcheng Laboratory（鹏城实验室）；The Chinese University of Hong Kong（香港中文大学）；Jilin University（吉林大学）；Southwest University（西南大学）；University of Electronic Science and Technology of China（电子科技大学）；Shenzhen University（深圳大学）
Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Models
School of Cyber Science and Technology, Shandong University（山东大学网络科学与技术学院）；Netflix Eyeline Studios；State Key Laboratory of Cryptography and Digital Economy Security, Shandong University（山东大学密码与数字经济安全国家重点实验室）；Shandong Key Laboratory of Artificial Intelligence Security, Shandong University（山东省人工智能安全重点实验室）
DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing
ShanghaiTech University（上海科技大学）；The State Key Laboratory of Blockchain and Data Security, Zhejiang University（浙江大学区块链与数据安全全国重点实验室）；The CoAI group, Tsinghua University（清华大学CoAI课题组）
Jailbreaking Vision-Language Models Through the Visual Modality
Warsaw University of Technology（华沙理工大学）；NASK National Research Institute（波兰国家研究研究院）；University of Liverpool（利物浦大学）；Indian Institute of Technology Roorkee（印度理工学院拉胡尔分校）；Toyota Technological Institute at Chicago（芝加哥丰田技术研究所）
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
Tsinghua University（清华大学）；Carnegie Mellon University（卡内基梅隆大学）；Zhongguancun Laboratory（中关村实验室）；National Financial Cryptography Research Center（国家金融密码学研究中心）；Shandong Institute of Blockchain（山东区块链研究院）；Shandong University（山东大学）
Red-teaming the Multimodal Reasoning: Jailbreaking Vision-Language Models via Cross-modal Entanglement Attacks
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；People’s Public Security University of China（中国人民公安大学）
Jailbreaking Large Language Models through Iterative Tool-Disguised Attacks via Reinforcement Learning
School of Cyberspace Science and Technology, Beijing Institute of Technology（信息科学与技术学院，北京理工大学）；School of Computer Science and Technology, Beijing Institute of Technology（计算机科学与技术学院，北京理工大学）；School of Computer Science, University of Auckland（计算机科学学院，奥克兰大学）；QAX Security Center, Qi-AnXin Technology Group Inc.（QAX安全中心，启安新科技集团）；Qi-AnXin Technology Group Inc.（启安新科技集团）；Zhongguancun Laboratory（中关村实验室）
