The State Key Laboratory of Blockchain and Data Security, Zhejiang University(区块链与数据安全国家重点实验室,浙江大学);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院);Nanyang Technological University(南洋理工大学);King Abdullah University of Science and Technology(国王阿卜杜勒阿齐兹科技大学);SUN YAT-SEN UNIVERSITY(孙中山大学)
Kedong Xiu, Yunhan Yang, Churui Zeng, Tianhang Zheng, Xinzhe Huang, Di Wang, Puning Zhao, Zhan Qin, Kui Ren
Existing gradient-based jailbreak attacks typically optimize a fixed-length adversarial suffix toward a predefined target response with a static optimization strategy. However, this fully static formulation undermines the effectiveness, efficiency and flexibility of gradient-based jailbreaking because (i) A predefined target usually lies in the low-probability region of a safety-aligned LLM's conditional output distribution, forcing the optimization to pursue an unlikely response pattern; (ii) Simple affirmative targets may even mislead LLMs to generate affirmative responses that are not highly relevant to the prompts; (iii) Fixed optimization strategy and suffix length treat all prompts equally, leading to limited attack capability for hard prompts and redundant capacity for easy ones. To address these limitations, we propose Dynamic Jailbreaking Attack (DJA), a parameter-free gradient-based jailbreak framework using dynamic candidate exploration, dynamic relevant targets and dynamic optimization strategy to craft adversarial prompts. In each optimization round, DJA samples multiple candidate target responses directly from the LLM's distribution conditioned on the current adversarial prompt. Among these candidates, DJA employs a multi-objective scorer to select an optimal target that satisfies multi-dimensional criteria such as harmfulness, relevance, and usefulness. Moreover, DJA introduces a parameter-free dynamic optimization strategy that allocates adversarial effort based on real-time feedback, adapting suffix length, candidate sampling capacity, and optimization iterations according to the difficulty of each harmful prompt. In an extensive evaluation of 40 safety-aligned LLMs (12 model families, scaling from 0.5B to 32B), DJA achieves a 100% ASR across all LLMs, requiring only 13.68 optimization rounds on average (10 iterations per round).
Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
School of EECS, Peking University(电子工程系,北京大学);School of Mathematical Sciences, Peking University(数学系,北京大学);State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(通用人工智能国家重点实验室,智能科学与技术学校,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学)
On Jailbreaking Quantized Language Models Through Fault Injection Attacks
Compumacy for Artificial Intelligence Solutions(人工智能解决方案公司);Electrical and Computer Engineering Department, George Mason University(乔治·马歇尔大学电气与计算机工程系);School of Electronics, Electrical Engineering and Computer Sciences, Queen’s University Belfast(女王大学贝尔法斯特电子、电气与计算机科学学院)
Activation-Guided Local Editing for Jailbreaking Attacks
Beihang University(北航);Hangzhou Innovation Institute(杭州创新研究院);HKUST(香港科技大学);South China University of Technology(华南理工大学);Nanyang Technological University(南洋理工大学);Kunming University of Science and Technology(昆明理工大学)
A Practical Framework for Evaluating Medical AI Security: Reproducible Assessment of Jailbreaking and Privacy Vulnerabilities Across Clinical Specialties
Geneshift: Impact of different scenario shift on Jailbreaking LLM
National University of Singapore(新加坡国立大学);NUS Graduate School(新加坡国立大学研究生院);Integrative Sciences and Engineering Programme(整合科学与工程项目);Institute of Data Science (IDS)(数据科学研究院);School of Computing(计算机学院)
from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学);Tsinghua University(清华大学)
OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
Institute of Software, Chinese Academy of Sciences, Beijing, China;Science \& Technology on Integrated Information System Laboratory, Beijing, China;State Key Laboratory of Complex System Modeling;University of Chinese Academy of Sciences, Beijing, China
Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking
School of Computer Science and Engineering, Sun Yat-Sen University(中山大学计算机科学与工程学院);Nanyang Technological University(南洋理工大学);Guangdong Province Key Laboratory of Information Security Technology(广东省信息安全技术重点实验室)
Does Chain-of-Thought Reasoning Really Reduce Harmfulness from Jailbreaking?
IIIS, Tsinghua University(清华大学信息学院);Shanghai Qi Zhi Institute(上海启智研究院);Department of Statistics and Data Science, University of Pennsylvania(宾夕法尼亚大学统计与数据科学系);Data Agent Group, Beijing Academy of Artificial Intelligence(北京人工智能研究院数据代理组)
Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreaking
Valencian Research Institute for Artificial Intelligence (VRAIN), Universitat Politècnica de València, Valencia, Spain.(瓦伦西亚人工智能研究 institute,瓦伦西亚理工大学,西班牙瓦伦西亚);Department of Computer Science, The University of Chicago, Chicago, USA(计算机科学系,芝加哥大学,美国芝加哥);Center for Automation and Robotics, Spanish National Research Council, Madrid, Spain(自动化与机器人中心,西班牙国家研究委员会,西班牙马德里)
Speechmatics(Speechmatics公司);MATS(机器学习对齐理论与实验项目(注:MATS全称为Machine Learning Alignment Theory and Experiments,为独立研究机构,此处保留缩写并补充说明));UCL(伦敦大学学院);Stanford University(斯坦福大学);University of Oxford(牛津大学);Tangentic(Tangentic公司);Anthropic(Anthropic公司)
Data to Defense: The Role of Curation in Customizing LLMs Against Jailbreaking Attacks
Stony Brook University(石溪大学);Binghamton University(宾汉姆顿大学);University of Iowa(爱荷华大学);Dartmouth College(达特茅斯学院);Georgia Institute of Technology(佐治亚理工学院)
Red-teaming the Multimodal Reasoning: Jailbreaking Vision-Language Models via Cross-modal Entanglement Attacks
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);People’s Public Security University of China(中国人民公安大学)
Jailbreaking Large Language Models through Iterative Tool-Disguised Attacks via Reinforcement Learning
School of Cyberspace Science and Technology, Beijing Institute of Technology(信息科学与技术学院,北京理工大学);School of Computer Science and Technology, Beijing Institute of Technology(计算机科学与技术学院,北京理工大学);School of Computer Science, University of Auckland(计算机科学学院,奥克兰大学);QAX Security Center, Qi-AnXin Technology Group Inc.(QAX安全中心,启安新科技集团);Qi-AnXin Technology Group Inc.(启安新科技集团);Zhongguancun Laboratory(中关村实验室)
Continuous Embedding Attacks via Clipped Inputs in Jailbreaking Large Language Models
University of New South Wales(新南威尔士大学);Delft University of Technology(代尔夫特理工大学);Nanyang Technological University(南洋理工大学);Huazhong University of Science and Technology(华中科技大学);Radboud University(拉德堡德大学)
PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualization
School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);RealAI;Tsinghua University(清华大学)
MaskForge: Structure-Aware Adaptive Attacks for Jailbreaking Diffusion Large Language Models
University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Johns Hopkins University(约翰霍普金斯大学);University of Southern California(南加州大学);Responsible AI Research (RAIR) Centre, The University of Adelaide(阿德莱德大学负责任人工智能研究中心)