The safety defense methods of Large language models(LLMs) stays limited because the dangerous prompts are manually curated to just few known attack types, which fails to keep pace with emerging varieties. Recent studies found that attaching suffixes to harmful instructions can hack the defense of LLMs and lead to dangerous outputs. However, similar to traditional text adversarial attacks, this approach, while effective, is limited by the challenge of the discrete tokens. This gradient based discrete optimization attack requires over 100,000 LLM calls, and due to the unreadable of adversarial suffixes, it can be relatively easily penetrated by common defense methods such as perplexity filters. To cope with this challenge, in this paper, we proposes an Adversarial Suffix Embedding Translation Framework (ASETF), aimed at transforming continuous adversarial suffix embeddings into coherent and understandable text. This method greatly reduces the computational overhead during the attack process and helps to automatically generate multiple adversarial samples, which can be used as data to strengthen LLMs security defense. Experimental evaluations were conducted on Llama2, Vicuna, and other prominent LLMs, employing harmful directives sourced from the Advbench dataset. The results indicate that our method significantly reduces the computation time of adversarial suffixes and achieves a much better attack success rate to existing techniques, while significantly enhancing the textual fluency of the prompts. In addition, our approach can be generalized into a broader method for generating transferable adversarial suffixes that can successfully attack multiple LLMs, even black-box LLMs, such as ChatGPT and Gemini.
Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?
Politecnico di Milano(米兰理工学院);Beihang University(北京航空航天大学);East China Normal University(华东师范大学);Fudan University(复旦大学);University of the Chinese Academy of Sciences(中国科学院大学);AI Security Lab(360人工智能安全实验室)
Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);Dept. Computing Science, Alberta Machine Intelligence Institute (Amii)(阿尔伯塔大学计算科学系、阿尔伯塔机器智能研究所);University of Alberta(阿尔伯塔大学);Tencent(腾讯)
A Troublemaker with Contagious Jailbreak Makes Chaos in Honest Towns
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China(认知与复杂系统决策智能重点实验室,自动化研究所,中国科学院,北京,中国);School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China(人工智能学院,中国科学院大学,北京,中国)
Continuous Embedding Attacks via Clipped Inputs in Jailbreaking Large Language Models
University of New South Wales(新南威尔士大学);Delft University of Technology(代尔夫特理工大学);Nanyang Technological University(南洋理工大学);Huazhong University of Science and Technology(华中科技大学);Radboud University(拉德堡德大学)
Toward Understanding the Transferability of Adversarial Suffixes in Large Language Models
Ludwig-Maximilians-Universität München(慕尼黑莱布尼茨-马克斯大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Carnegie Mellon University(卡内基梅隆大学);University of Maryland(马里兰大学)
AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
Dept. of Comp. Sci. and Tech., Inst. for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, 100084, China(计算机科学与技术系、人工智能研究所、清华-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学、北京);Department of Mathematical Sciences, Tsinghua University(数学科学系、清华大学);BIMSA, Beijing, China(BIMSA、北京、中国)
TRAP: Targeted Random Adversarial Prompt Honeypot for Black-Box Identification
Parameter Lab(参数实验室);IT University of Copenhagen(哥本哈根信息技术大学);Pioneer Centre for Artificial Intelligence(人工智能先锋中心);NAVER AI Lab(NAVER人工智能实验室);University of Tübingen(蒂宾根大学)
Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings
Graduate School of Informatics, Osaka Metropolitan University(大阪公立大学情报学研究科);BRAC University(BRAC大学);Bangladesh University of Engineering and Technology (BUET)(孟加拉工程技术大学);New Mexico State University(新墨西哥州立大学)