Large language models trained for safety and harmlessness remain susceptible to adversarial misuse, as evidenced by the prevalence of "jailbreak" attacks on early releases of ChatGPT that elicit undesired behavior. Going beyond recognition of the issue, we investigate why such attacks succeed and how they can be created. We hypothesize two failure modes of safety training: competing objectives and mismatched generalization. Competing objectives arise when a model's capabilities and safety goals conflict, while mismatched generalization occurs when safety training fails to generalize to a domain for which capabilities exist. We use these failure modes to guide jailbreak design and then evaluate state-of-the-art models, including OpenAI's GPT-4 and Anthropic's Claude v1.3, against both existing and newly designed attacks. We find that vulnerabilities persist despite the extensive red-teaming and safety-training efforts behind these models. Notably, new attacks utilizing our failure modes succeed on every prompt in a collection of unsafe requests from the models' red-teaming evaluation sets and outperform existing ad hoc jailbreaks. Our analysis emphasizes the need for safety-capability parity -- that safety mechanisms should be as sophisticated as the underlying model -- and argues against the idea that scaling alone can resolve these safety failure modes.
Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation
University of Technology Sydney, Sydney, Australia（技术悉尼大学）；School of Cyber Science and Engineering, Southeast University, Nanjing, China（东南大学计算机科学与工程学院）；Xi'an Jiaotong University, Xi'an, China（西安交通大学）
When Skills Meet Safety: Benchmarking and Characterizing the Adaptive Jailbreak Robustness of Skill-Merged LLMs
Google（谷歌公司）；University of New South Wales（新南威尔士大学）；University of Technology Sydney（悉尼科技大学）；Zhejiang University（浙江大学）；Australian National University（澳大利亚国立大学）
SafeRedirect: Defeating Internal Safety Collapse via Task-Completion Redirection in Frontier LLMs
Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen 518055, China（南方科技大学计算机科学与工程系，深圳518055，中国）；The Hong Kong Polytechnic University, Hong Kong, China（香港理工大学，香港，中国）；George Washington University（乔治华盛顿大学）；School of Data Science, Lingnan University, Hong Kong, China（岭南大学数据科学学院，香港，中国）
Safer by Diffusion, Broken by Context: Diffusion LLM's Safety Blessing and Its Failure Mode
Torr Vision Group, University of Oxford（牛津大学Torr视觉组）；School of Data Science, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）数据科学学院）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Microsoft（微软）
AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents
Gray Swan AI（格雷天鹅人工智能公司）；UK AI Security Institute（英国人工智能安全研究所）；EPFL（洛桑联邦理工学院）；Center for AI Safety（人工智能安全中心）；Carnegie Mellon University（卡内基梅隆大学）；University of Oxford（牛津大学）
Supporting Safety Analysis of Image-processing DNNs through Clustering-based Approaches
SnT Centre, University of Luxembourg（卢森堡大学SnT中心）；University of Luxembourg（卢森堡大学）；University of Ottawa（渥太华大学）；Lero Centre, University of Limerick（利默里克大学Lero中心）；School of EECS, University of Ottawa（渥太华大学电子工程与计算机科学学院）
