近期研究提出了自动化红队测试方法,用于检测目标大语言模型(LLM)的安全漏洞。这类方法借助红队LLM挖掘能诱发目标LLM产生有害行为的输入。本文研究可实现定向安全评估的红队测试策略,提出一种带邻近约束的红队优化框架,要求挖掘出的提示词必须与给定数据集的参考提示词相似。该数据集作为挖掘提示词的模板,将测试用例的搜索锚定在特定主题、写作风格或有害行为类型上。研究发现,现有自回归模型架构在该场景下表现不佳,因此本文提出一种受文本扩散模型启发的黑盒红队方法:Diffusion for Auditing and Red-Teaming(DART,审计与红队扩散模型)。DART通过在嵌入空间中扰动参考提示词对其进行修改,可直接控制引入的改动量。本文通过将该方法的效果与基于模型微调、零样本及少样本提示的现有方法进行对比,开展了系统评估。结果表明,DART在挖掘与参考提示词高度邻近的有害输入方面,效果显著更优。
英文摘要:
Recent work has proposed automated red-teaming methods for testing the vulnerabilities of a given target large language model (LLM). These methods use red-teaming LLMs to uncover inputs that induce harmful behavior in a target LLM. In this paper, we study red-teaming strategies that enable a targeted security assessment. We propose an optimization framework for red-teaming with proximity constraints, where the discovered prompts must be similar to reference prompts from a given dataset. This dataset serves as a template for the discovered prompts, anchoring the search for test-cases to specific topics, writing styles, or types of harmful behavior. We show that established auto-regressive model architectures do not perform well in this setting. We therefore introduce a black-box red-teaming method inspired by text-diffusion models: Diffusion for Auditing and Red-Teaming (DART). DART modifies the reference prompt by perturbing it in the embedding space, directly controlling the amount of change introduced. We systematically evaluate our method by comparing its effectiveness with established methods based on model fine-tuning and zero- and few-shot prompting. Our results show that DART is significantly more effective at discovering harmful inputs in close proximity to the reference prompt.
RedTopic: Toward Topic-Diverse Red Teaming of Large Language Models
Fudan University(复旦大学);City University of Hong Kong(香港城市大学);Deakin University(德克萨斯大学);Hon Hai Research Institute(鸿海研究室);Hong Kong University of Science and Technology(香港理工大学)
X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents
University of California, Los Angeles(加州大学洛杉矶分校);University of Washington(华盛顿大学);Qatar Computing Research Institute(卡塔尔计算研究所);Google(谷歌);Stanford University(斯坦福大学)
Learning diverse attacks on large language models for robust red-teaming and safety tuning
KAIST(韩国科学技术院);Mila – Québec AI Institute(米拉-魁北克人工智能研究所);Université de Montréal(蒙特利尔大学);McGill University(麦吉尔大学);National University of Singapore(新加坡国立大学);University of Edinburgh(爱丁堡大学);CIFAR(加拿大高级研究所)
Arondight: Red Teaming Large Vision Language Models with Auto-generated Multi-modal Jailbreak Prompts
City University of Hong Kong(香港城市大学);City University of Hong Kong (Dongguan)(香港城市大学(东莞));University of Science and Technology Beijing(北京科技大学);The University of Melbourne(墨尔本大学)
The Missing Half: Unveiling Training-time Implicit Safety Risks Beyond Deployment
The Conversational AI (CoAI) group, DCST, Tsinghua University(清华大学智能对话研究院);National University of Singapore(新加坡国立大学);Pattern Recognition Center, WeChat AI, Tencent Inc, China(腾讯人工智能研究院)
Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs
Georgia Institute of Technology(佐治亚理工学院);University of Bristol(布里斯托大学);University of Maryland(马里兰大学);University College London(伦敦大学学院);MATS;Astra;MIT CSAIL(麻省理工学院计算机科学与人工智能实验室)
Red-Teaming Text-to-Image Systems by Rule-based Preference Modeling
KLMM, UCAS, Academy of Mathematics and Systems Science, Chinese Academy of Sciences(UCAS信息科学学院、数学与系统科学研究院、中国科学院);College of AI, Tsinghua University(清华大学人工智能学院)
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家实验室,南京大学);School of Artificial Intelligence, Nanjing University(人工智能学院,南京大学);The Hong Kong University of Science and Technology(香港科技大学);The Chinese University of Hong Kong, Shenzhen(香港大学深圳校区);Southern University of Science and Technology(南方科技大学)
Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundary
The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security(杭州高新技术区(滨江)区块链与数据安全研究院);Language and Machine Intelligence Department, Ant Group(蚂蚁集团语言与机器智能部门)
Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generation
Harvard University(哈佛大学);Google Research(谷歌研究院);University of Zurich(苏黎世大学);Carnegie Mellon University(卡内基梅隆大学);University of Oxford(牛津大学);University College London(伦敦大学学院);Cohere(科here公司);MLCommons(MLCommons组织)
Negation Blindness in Large Language Models: Unveiling the NO Syndrome in Image Generation
Aligarh Muslim University(阿里格尔穆斯林大学);VIT Bhopal University(VIT博帕尔大学);Nanyang Technological University(南洋理工大学);Technical University of Munich(慕尼黑工业大学);Imperial College London(伦敦帝国学院);Edinburgh Napier University(爱丁堡龙比亚大学)
The Art of the Jailbreak: Formulating Jailbreak Attacks for LLM Security Beyond Binary Scoring
Department of Computer Science, University of Texas at El Paso(德克萨斯大学埃尔帕索分校计算机科学系);School of Computing, Southern Illinois University Carbondale(南方伊利诺伊大学卡罗尔丹分校计算学院);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)