The rapid progress in open-source large language models (LLMs) is significantly advancing AI development. Extensive efforts have been made before model release to align their behavior with human values, with the primary goal of ensuring their helpfulness and harmlessness. However, even carefully aligned models can be manipulated maliciously, leading to unintended behaviors, known as "jailbreaks". These jailbreaks are typically triggered by specific text inputs, often referred to as adversarial prompts. In this work, we propose the generation exploitation attack, an extremely simple approach that disrupts model alignment by only manipulating variations of decoding methods. By exploiting different generation strategies, including varying decoding hyper-parameters and sampling methods, we increase the misalignment rate from 0% to more than 95% across 11 language models including LLaMA2, Vicuna, Falcon, and MPT families, outperforming state-of-the-art attacks with $30\times$ lower computational cost. Finally, we propose an effective alignment method that explores diverse generation strategies, which can reasonably reduce the misalignment rate under our attack. Altogether, our study underscores a major failure in current safety evaluation and alignment procedures for open-source LLMs, strongly advocating for more comprehensive red teaming and better alignment before releasing such models. Our code is available at https://github.com/Princeton-SysML/Jailbreak_LLM.
Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Dept. Computing Science, Alberta Machine Intelligence Institute (Amii)(阿尔伯塔大学计算科学系、阿尔伯塔机器智能研究所);University of Alberta(阿尔伯塔大学);Tencent(腾讯)
A Simple and Efficient Jailbreak Method Exploiting LLMs' Helpfulness
The Harbin Institute of Technology(哈尔滨工业大学);The Hong Kong Polytechnic University(香港理工大学);Shenzhen University(深圳大学);Shenzhen Loop Area Institute(深圳河套学院)
PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualization
School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院);School of Electronic and Computer Engineering, Peking University(北京大学电子与计算机工程学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The University of Hong Kong(香港大学);Fudan University(复旦大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Dual-Space Smoothness for Robust and Balanced LLM Unlearning
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);Department of Computer Science and Engineering, University of Notre Dame(圣母大学计算机科学与工程系)
Jailbreaking Vision-Language Models Through the Visual Modality
Warsaw University of Technology(华沙理工大学);NASK National Research Institute(波兰国家研究研究院);University of Liverpool(利物浦大学);Indian Institute of Technology Roorkee(印度理工学院拉胡尔分校);Toyota Technological Institute at Chicago(芝加哥丰田技术研究所)
Learning Wisdom from Errors: Promoting LLM's Continual Relation Learning through Exploiting Error Cases
Beijing University of Posts and Telecommunications(北京邮电大学);University of Electronic Science and Technology of China(电子科技大学);China National Institute of Standardization(中国标准化研究院)
The Forgotten Shield: Safety Grafting in Parameter-Space for Medical MLLMs
National University of Defense Technology(国防科技大学);Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统(MAIS));Multimedia Department, Xiaomi Inc(小米公司多媒体部门);Centre for Artificial Intelligence and Robotics, Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences, Hong Kong(香港科学院人工智能与机器人中心);School of Artificial Intelligence, University of Chinese Academy of Sciences, UCAS(中国科学院大学人工智能学院)
A Theoretical Understanding of Self-Correction through In-context Alignment
School of EECS, Peking University(北京大学电子工程与计算机科学学院);School of Mathematical Sciences, Peking University(北京大学数学科学学院);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);TU Munich(慕尼黑工业大学);Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院)
Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solving
Florida International University(佛罗里达国际大学);SRI International(SRI国际(斯坦福国际研究院));United States Military Academy(美国军事学院(西点军校));University of Colorado Boulder(科罗拉多大学博尔德分校);University of Central Florida(中佛罗里达大学);Vanderbilt University(范德堡大学)