While Large Language Models (LLMs) have achieved tremendous success in various applications, they are also susceptible to jailbreaking attacks. Several primary defense strategies have been proposed to protect LLMs from producing harmful information, mostly focusing on model fine-tuning or heuristical defense designs. However, how to achieve intrinsic robustness through prompt optimization remains an open problem. In this paper, motivated by adversarial training paradigms for achieving reliable robustness, we propose an approach named Prompt Adversarial Tuning (PAT) that trains a prompt control attached to the user prompt as a guard prefix. To achieve our defense goal whilst maintaining natural performance, we optimize the control prompt with both adversarial and benign prompts. Comprehensive experiments show that our method is effective against both grey-box and black-box attacks, reducing the success rate of advanced attacks to nearly 0%, while maintaining the model's utility on the benign task and incurring only negligible computational overhead, charting a new perspective for future explorations in LLM security. Our code is available at https://github.com/PKU-ML/PAT.
Adversarial attacks optimize against models to defeat defenses. Existing defenses are static, and stay the same once trained, even while attacks change. We argue that models should fight back, and optimize their defenses against attacks at test time. We propose dynamic defenses, to adapt the model and input during testing, by defensive entropy minimization (dent). Dent alters testing, but not training, for compatibility with existing models and train-time defenses. Dent improves the robustness of adversarially-trained defenses and nominally-trained models against white-box, black-box, and adap…
Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
Hunan University Changsha, PRC；Georgia Institute of Technology Atlanta, USA；Kyoto University Kyoto, Japan；Purdue University West Lafayette, USA；National Taiwan Normal University Taipei, ROC；University of Liverpool Suzhou, PRC；Hong Kong University of Science；University of Hawaii Honolulu, USA；The University of Texas at Dallas Dallas, USA；University of Wisconsin-Madison Madison, USA；Emory University Atlanta, USA；College of William \& Mary Williamsburg, USA
Before the Script, Set the Stage: How Worldview Simulation Amplifies Psychologically Grounded Persuasion in Multi-Turn Jailbreaking
AI4S Center, Shanghai Qi Zhi Institute（上海期智研究院AI4S中心）；College of AI, Tsinghua University（清华大学人工智能学院）；Fangcun AI（方寸人工智能）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）
Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
Georgia Institute of Technology（佐治亚理工学院）；University of Bristol（布里斯托大学）；University of Maryland（马里兰大学）；University College London（伦敦大学学院）；MATS；Astra；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）
