本文提出 DATDP 方法，通过反复调用评估 LLM 检测提示中的危险行为和越狱尝试，成功阻断 Best-of-N 越狱攻击，且小模型同样有效，可低成本提升生成式 AI 安全性。
摘要：
近期研究表明，利用随机增强（如大小写、标点等）反复使用的 Best-of-N（BoN）越狱攻击对所有主流大语言模型（LLMs）均有效。我们发现，BoN 论文中 100% 的成功越狱（置信区间 [99.65%, 100.00%]）以及我们复现实验中 99.8% 的成功越狱（置信区间 [99.28%, 99.98%]）均被我们的 Defense Against The Dark Prompts（DATDP）方法所阻断。DATDP 算法通过反复利用一个评估 LLM 来评估提示是否具有危险或操纵性行为——与其他一些方法不同，DATDP 还明确检测越狱尝试——直到生成一个稳健的安全评级。即使使用较小的 LLM 来驱动评估，这一成功仍然持续（Claude 和 LLaMa-3-8B-instruct 被证明能力几乎相当）。这些结果表明，尽管语言模型对输入中看似无害的变化很敏感，但它们似乎也能够成功评估这些输入的危险性。因此，DATDP 的各种版本可以低成本地添加到生成式 AI 系统中，从而立即显著提升安全性。
英文摘要：
Recent work showed Best-of-N (BoN) jailbreaking using repeated use of random augmentations (such as capitalization, punctuation, etc) is effective against all major large language models (LLMs). We have found that $100\%$ of the BoN paper's successful jailbreaks (confidence interval $[99.65\%, 100.00\%]$) and $99.8\%$ of successful jailbreaks in our replication (confidence interval $[99.28\%, 99.98\%]$) were blocked with our Defense Against The Dark Prompts (DATDP) method. The DATDP algorithm works by repeatedly utilizing an evaluation LLM to evaluate a prompt for dangerous or manipulative behaviors--unlike some other approaches, DATDP also explicitly looks for jailbreaking attempts--until a robust safety rating is generated. This success persisted even when utilizing smaller LLMs to power the evaluation (Claude and LLaMa-3-8B-instruct proved almost equally capable). These results show that, though language models are sensitive to seemingly innocuous changes to inputs, they seem also capable of successfully evaluating the dangers of these inputs. Versions of DATDP can therefore be added cheaply to generative AI systems to produce an immediate significant increase in safety.
Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
School of EECS, Peking University（电子工程系，北京大学）；School of Mathematical Sciences, Peking University（数学系，北京大学）；State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University（通用人工智能国家重点实验室，智能科学与技术学校，北京大学）；Institute for Artificial Intelligence, Peking University（人工智能研究院，北京大学）
QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Language
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Pengcheng Laboratory（鹏城实验室）；The Chinese University of Hong Kong（香港中文大学）；Jilin University（吉林大学）；Southwest University（西南大学）；University of Electronic Science and Technology of China（电子科技大学）；Shenzhen University（深圳大学）
Activation-Guided Local Editing for Jailbreaking Attacks
Beihang University（北航）；Hangzhou Innovation Institute（杭州创新研究院）；HKUST（香港科技大学）；South China University of Technology（华南理工大学）；Nanyang Technological University（南洋理工大学）；Kunming University of Science and Technology（昆明理工大学）
SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner
The Hong Kong University of Science and Technology（香港科技大学）；University of Oregon（俄勒冈大学）；Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）；HSBC（汇丰银行）
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
Dept. of Comp. Sci. and Tech., Inst. for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, 100084, China（计算机科学与技术系、人工智能研究所、清华-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学、北京）；Department of Mathematical Sciences, Tsinghua University（数学科学系、清华大学）；BIMSA, Beijing, China（BIMSA、北京、中国）
Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreaking
School of Computer Science and Engineering, Sun Yat-Sen University（中山大学计算机科学与工程学院）；Nanyang Technological University（南洋理工大学）；Guangdong Province Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）
Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings
University of Notre Dame（圣母大学）；Huazhong University of Science and Technology（华中科技大学）；Tsinghua University（清华大学）；Lehigh University（里海大学）；University of Illinois Chicago（芝加哥大学伊利诺伊分校）
Feint and Attack: Attention-Based Strategies for Jailbreaking and Protecting LLMs
Beijing University of Posts and Telecommunications（北京邮电大学）；Hangzhou Dianzi University（杭州电子科技大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；Fujian Cancer Hospital（福建癌症医院）
A Practical Framework for Evaluating Medical AI Security: Reproducible Assessment of Jailbreaking and Privacy Vulnerabilities Across Clinical Specialties
School of Computer Science and Technology, Guangdong University of Technology（广东工业大学计算机科学与技术学院）；School of Computer Science and Intelligence Education, Lingnan Normal University（岭南师范学院计算机与智能教育学院）；College of Computer, National University of Defense Technology（国防科技大学计算机学院）；School of Information Science and Technology, Guangdong University of Foreign Studies（广东外语外贸大学信息科学与技术学院）
Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion
Department of Computer & Information Science & Engineering, University of Florida（佛罗里达大学计算机与信息科学与工程系）；School of Computer Science, University of Oklahoma（俄克拉荷马大学计算机科学学院）；Computer Science Laboratory, SRI International（SRI国际计算机科学实验室）
DualBreach: Efficient Dual-Jailbreaking via Target-Driven Initialization and Multi-Target Optimization
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China（区块链与数据安全国家重点实验室，浙江大学，杭州，中国）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China（杭州高新技术区（滨江）区块链与数据安全研究院，杭州，中国）
OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing
Institute of Software, Chinese Academy of Sciences, Beijing, China；Science \& Technology on Integrated Information System Laboratory, Beijing, China；State Key Laboratory of Complex System Modeling；University of Chinese Academy of Sciences, Beijing, China
Speechmatics（Speechmatics公司）；MATS（机器学习对齐理论与实验项目（注：MATS全称为Machine Learning Alignment Theory and Experiments，为独立研究机构，此处保留缩写并补充说明））；UCL（伦敦大学学院）；Stanford University（斯坦福大学）；University of Oxford（牛津大学）；Tangentic（Tangentic公司）；Anthropic（Anthropic公司）
EasyJailbreak: A Unified Framework for Jailbreaking Large Language Models
School of Computer Science, Fudan University（复旦大学计算机科学学院）；Shanghai AI Laboratory（上海人工智能实验室）；Institute of Modern Languages and Linguistics, Fudan University（复旦大学现代语言与语言学学院）
Jailbreaking Multimodal Large Language Models using Multi-Clip Video
Department of Applied Artificial Intelligence, Sungkyunkwan University（应用人工智能系，成均馆大学）；Department of Human-Artificial Intelligence Interaction, Sungkyunkwan University（人机交互系，成均馆大学）
