Toward Understanding the Transferability of Adversarial Suffixes in Large Language Models
Ludwig-Maximilians-Universität München(慕尼黑莱布尼茨-马克斯大学);Munich Center for Machine Learning (MCML)(慕尼黑机器学习中心);Carnegie Mellon University(卡内基梅隆大学);University of Maryland(马里兰大学)
Sarah Ball, Niki Hasrati, Alexander Robey, Avi Schwarzschild, Frauke Kreuter, Zico Kolter, Andrej Risteski
Discrete optimization-based jailbreaking attacks on large language models aim to generate short, nonsensical suffixes that, when appended onto input prompts, elicit disallowed content. Notably, these suffixes are often transferable -- succeeding on prompts and models for which they were never optimized. And yet, despite the fact that transferability is surprising and empirically well-established, the field lacks a rigorous analysis of when and why transfer occurs. To fill this gap, we identify three statistical properties that strongly correlate with transfer success across numerous experimental settings: (1) how much a prompt without a suffix activates a model's internal refusal direction, (2) how strongly a suffix induces a push away from this direction, and (3) how large these shifts are in directions orthogonal to refusal. On the other hand, we find that prompt semantic similarity only weakly correlates with transfer success. These findings lead to a more fine-grained understanding of transferability, which we use in interventional experiments to showcase how our statistical analysis can translate into practical improvements in attack success.
Universal Adversarial Suffixes for Language Models Using Reinforcement Learning with Calibrated Reward
Center for Intelligent Cyber Physical Systems, Indian Institute of Technology Guwahati, India(智能网络物理系统中心,印度理工学院古瓦哈提);Department of Computer Science and Engineering, Indian Institute of Technology Guwahati, India(计算机科学与工程系,印度理工学院古瓦哈提)
Universal Adversarial Suffixes Using Calibrated Gumbel-Softmax Relaxation
Center for Intelligent Cyber Physical Systems(智能网络物理系统中心);Indian Institute of Technology Guwahati(印度古瓦哈提理工学院);Department of Computer Science and Engineering(计算机科学与工程系)
Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?
Politecnico di Milano(米兰理工学院);Beihang University(北京航空航天大学);East China Normal University(华东师范大学);Fudan University(复旦大学);University of the Chinese Academy of Sciences(中国科学院大学);AI Security Lab(360人工智能安全实验室)
CT-GAT: Cross-Task Generative Adversarial Attack based on Transferability
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
On the Transferability of Adversarial Attacksagainst Neural Text Classifier
School of Computer Science, Fudan University(复旦大学计算机学院);Shanghai Key Laboratory of Intelligent Information Processing(上海市智能信息处理重点实验室);Department of Computer Science, University of California, Los Angeles(加利福尼亚大学洛杉矶分校计算机科学系)