The adoption of large language models (LLMs) in many applications, from customer service chat bots and software development assistants to more capable agentic systems necessitates research into how to secure these systems. Attacks like prompt injection and jailbreaking attempt to elicit responses and actions from these models that are not compliant with the safety, privacy, or content policies of organizations using the model in their application. In order to counter abuse of LLMs for generating potentially harmful replies or taking undesirable actions, LLM owners must apply safeguards during training and integrate additional tools to block the LLM from generating text that abuses the model. Jailbreaking prompts play a vital role in convincing an LLM to generate potentially harmful content, making it important to identify jailbreaking attempts to block any further steps. In this work, we propose a novel approach to detect jailbreak prompts based on pairing text embeddings well-suited for retrieval with traditional machine learning classification algorithms. Our approach outperforms all publicly available methods from open source LLM security applications.
Continuous Embedding Attacks via Clipped Inputs in Jailbreaking Large Language Models
University of New South Wales（新南威尔士大学）；Delft University of Technology（代尔夫特理工大学）；Nanyang Technological University（南洋理工大学）；Huazhong University of Science and Technology（华中科技大学）；Radboud University（拉德堡德大学）
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
Dept. of Comp. Sci. and Tech., Inst. for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, 100084, China（计算机科学与技术系、人工智能研究所、清华-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学、北京）；Department of Mathematical Sciences, Tsinghua University（数学科学系、清华大学）；BIMSA, Beijing, China（BIMSA、北京、中国）
Adaptive Prompt Embedding Optimization for LLM Jailbreaking
School of Information Studies, McGill University, Montreal, QC, Canada（麦吉尔大学信息研究学院）；Department of Computer Science, Kean University, Union, NJ, United States（凯恩大学计算机科学系）；Mila - Quebec AI Institute, Montreal, QC, Canada（魁北克人工智能研究所）；Faculty of Information, University of Toronto, Toronto, ON, Canada（多伦多大学信息学院）
PiCo: Jailbreaking Multimodal Large Language Models via Pictorial Code Contextualization
School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts
Tsinghua University（清华大学）；Carnegie Mellon University（卡内基梅隆大学）；Zhongguancun Laboratory（中关村实验室）；National Financial Cryptography Research Center（国家金融密码学研究中心）；Shandong Institute of Blockchain（山东区块链研究院）；Shandong University（山东大学）
Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models
School of Computing and Information Systems, The University of Melbourne, Australia（墨尔本大学计算机与信息系）；School of Computing and Information Systems, Singapore Management University, Singapore（新加坡管理大学计算机与信息系）；School of Information Technology, Deakin University, Australia（德肯大学信息科技系）；Institute of Trustworthy Embodied AI, Fudan University, China（复旦大学可信具身人工智能研究所）
Mitsubishi Electric Corporation（三菱电机株式会社）；Pennsylvania State University（宾夕法尼亚州立大学）；The Ohio State University（俄亥俄州立大学）；Mitsubishi Electric Research Laboratories（三菱电机研究所）
Latent-space adversarial training with post-aware calibration for defending large language models against jailbreak attacks
Lab of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China（教育人工智能实验室，东华大学，上海200062，中国）；Shanghai Institute of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China（教育人工智能研究所，东华大学，上海200062，中国）；School of Computer Science and Technology, East China Normal University, Shanghai 200062, China（计算机科学与技术学院，东华大学，上海200062，中国）
