arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.10012cs.LG
收藏

Ring-A-Bell!扩散模型的概念移除方法有多可靠?

Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?

National Yang Ming Chiao Tung University(国立阳明交通大学);University of Illinois at Urbana Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Chicago(芝加哥大学);IBM Research(IBM研究院)

Yu-Lin Tsai, Chia-Yi Hsu, Chulin Xie, Chih-Hsun Lin, Jia-You Chen, Bo Li, Pin-Yu Chen, Chia-Mu Yu, Chun-Ying Huang

AI总结:

本文提出模型无关的红队测试工具 Ring-A-Bell,通过概念提取与安全提示词操纵,揭示扩散模型概念移除和安全过滤机制在面对变换后提示词时的可靠性缺陷。

摘要:

用于文本到图像(T2I)合成的扩散模型,如 Stable Diffusion(SD),近期在生成高质量内容方面展现出卓越能力。然而,这一进展也引发了对潜在滥用的若干担忧,尤其是在生成受版权保护、被禁止和受限内容,或 NSFW(不适宜工作场所)图像方面。尽管已有努力通过评估阶段实施安全过滤器或微调模型以消除不良概念或风格来缓解此类问题,但这些安全措施在处理广泛提示词时的有效性在很大程度上仍未得到探索。本文旨在通过提出一种新颖的概念检索评估算法来研究这些安全机制。我们引入了 Ring-A-Bell,这是一种面向 T2I 扩散模型的模型无关红队测试工具,整个评估可以提前准备,无需事先了解目标模型。具体而言,Ring-A-Bell 首先执行概念提取,以获得敏感和不适当概念的整体表示。随后,通过利用提取出的概念,Ring-A-Bell 自动识别扩散模型的问题提示词,并生成相应的不适当内容,使用户能够评估已部署安全机制的可靠性。最后,我们通过测试 Midjourney 等在线服务以及各种概念移除方法,对所提方法进行了实证验证。结果表明,Ring-A-Bell 通过操纵安全提示词基准,能够将原本被视为安全的提示词转化为可绕过现有安全机制的提示词,从而揭示所谓安全机制的缺陷,这些缺陷在实践中可能导致有害内容的生成。我们的代码可在 https://github.com/chiayi-hsu/Ring-A-Bell 获取。

英文摘要:

Diffusion models for text-to-image (T2I) synthesis, such as Stable Diffusion (SD), have recently demonstrated exceptional capabilities for generating high-quality content. However, this progress has raised several concerns of potential misuse, particularly in creating copyrighted, prohibited, and restricted content, or NSFW (not safe for work) images. While efforts have been made to mitigate such problems, either by implementing a safety filter at the evaluation stage or by fine-tuning models to eliminate undesirable concepts or styles, the effectiveness of these safety measures in dealing with a wide range of prompts remains largely unexplored. In this work, we aim to investigate these safety mechanisms by proposing one novel concept retrieval algorithm for evaluation. We introduce Ring-A-Bell, a model-agnostic red-teaming tool for T2I diffusion models, where the whole evaluation can be prepared in advance without prior knowledge of the target model. Specifically, Ring-A-Bell first performs concept extraction to obtain holistic representations for sensitive and inappropriate concepts. Subsequently, by leveraging the extracted concept, Ring-A-Bell automatically identifies problematic prompts for diffusion models with the corresponding generation of inappropriate content, allowing the user to assess the reliability of deployed safety mechanisms. Finally, we empirically validate our method by testing online services such as Midjourney and various methods of concept removal. Our results show that Ring-A-Bell, by manipulating safe prompting benchmarks, can transform prompts that were originally regarded as safe to evade existing safety mechanisms, thus revealing the defects of the so-called safety mechanisms which could practically lead to the generation of harmful contents. Our codes are available at https://github.com/chiayi-hsu/Ring-A-Bell.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑