arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.18510cs.CL
收藏

大规模野外团队测试:从真实世界越狱到(对抗性)更安全的语言模型

WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models

Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Seoul National University(首尔大学);Carnegie Mellon University(卡内基梅隆大学);University of Washington(华盛顿大学)

Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, Nouha Dziri

AI总结:

本文提出WildTeaming框架,通过挖掘野外用户交互发现5.7K种越狱策略,并构建262K规模的WildJailbreak数据集,实现安全训练中保护与能力平衡。

摘要:

我们引入了WildTeaming,一个自动化的LLM安全红队测试框架,该框架挖掘野外用户与聊天机器人的交互,以发现5.7K个新颖越狱策略的独特簇,然后组合多种策略以系统性地探索新颖越狱。与先前通过招募人类工作者、基于梯度的优化或使用LLM进行迭代修订来进行红队测试的工作相比,我们的工作调查了那些未被特别指示破坏系统的聊天机器人用户的越狱行为。WildTeaming揭示了前沿LLM先前未识别的漏洞,与最先进的越狱方法相比,产生了多达4.6倍更多样化和成功的对抗性攻击。虽然存在许多用于越狱评估的数据集,但用于越狱训练的开源数据集却很少,因为即使模型权重是开放的,安全训练数据也一直是封闭的。通过WildTeaming,我们创建了WildJailbreak,一个大规模的开源合成安全数据集,包含262K个普通(直接请求)和对抗性(复杂越狱)提示-响应对。为了减轻过度安全行为,WildJailbreak提供了两种对比类型的查询:1)有害查询(普通和对抗性)和2)良性查询,这些查询在形式上类似于有害查询但不包含任何伤害。由于WildJailbreak显著提升了现有安全资源的质量和规模,它独特地使我们能够检查数据的扩展效应以及安全训练中数据属性与模型能力之间的相互作用。通过大量实验,我们确定了能够实现安全行为理想平衡的训练属性:适当的保护而不过度拒绝,有效处理普通和对抗性查询,以及通用能力的最小化(如果有的话)下降。WildJailbreak的所有组成部分都有助于实现模型的平衡安全行为。

英文摘要:

We introduce WildTeaming, an automatic LLM safety red-teaming framework that mines in-the-wild user-chatbot interactions to discover 5.7K unique clusters of novel jailbreak tactics, and then composes multiple tactics for systematic exploration of novel jailbreaks. Compared to prior work that performed red-teaming via recruited human workers, gradient-based optimization, or iterative revision with LLMs, our work investigates jailbreaks from chatbot users who were not specifically instructed to break the system. WildTeaming reveals previously unidentified vulnerabilities of frontier LLMs, resulting in up to 4.6x more diverse and successful adversarial attacks compared to state-of-the-art jailbreak methods. While many datasets exist for jailbreak evaluation, very few open-source datasets exist for jailbreak training, as safety training data has been closed even when model weights are open. With WildTeaming we create WildJailbreak, a large-scale open-source synthetic safety dataset with 262K vanilla (direct request) and adversarial (complex jailbreak) prompt-response pairs. To mitigate exaggerated safety behaviors, WildJailbreak provides two contrastive types of queries: 1) harmful queries (vanilla & adversarial) and 2) benign queries that resemble harmful queries in form but contain no harm. As WildJailbreak considerably upgrades the quality and scale of existing safety resources, it uniquely enables us to examine the scaling effects of data and the interplay of data properties and model capabilities during safety training. Through extensive experiments, we identify the training properties that enable an ideal balance of safety behaviors: appropriate safeguarding without over-refusal, effective handling of vanilla and adversarial queries, and minimal, if any, decrease in general capabilities. All components of WildJailbeak contribute to achieving balanced safety behaviors of models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑