arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.16807cs.CV
收藏

STEREO:一种针对文本到图像扩散模型对抗鲁棒概念擦除的两阶段框架

STEREO: A Two-Stage Framework for Adversarially Robust Concept Erasing from Text-to-Image Diffusion Models

Johns Hopkins University(约翰斯·霍普金斯大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学);Khalifa University(哈利法大学);Michigan State University(密歇根州立大学)

Koushik Srivatsan, Fahad Shamshad, Muzammal Naseer, Vishal M. Patel, Karthik Nandakumar

AI总结:

提出两阶段框架STEREO,通过对抗训练识别嵌入空间漏洞并结合基于锚概念的单阶段微调,在保持模型实用性的同时实现了对文本到图像扩散模型概念擦除的对抗鲁棒性。

摘要:

大规模文本到图像扩散(T2ID)模型的快速普及引发了人们对其在生成有害内容方面可能被滥用的严重担忧。尽管已经提出了许多从T2ID模型中擦除不良概念的方法,但它们往往提供一种虚假的安全感;概念擦除模型(CEMs)仍然可以通过对抗攻击被操纵以重新生成被擦除的概念。虽然最近出现了一些基于对抗训练的鲁棒概念擦除方法,但它们为了实现鲁棒性而牺牲了实用性(良性概念的生成质量),和/或仍然容易受到高级嵌入空间攻击。这些局限性源于鲁棒CEMs未能彻底搜索嵌入空间中的“盲点”。为了弥补这一差距,我们提出了STEREO,一种新颖的两阶段框架,它将对抗训练作为第一步,而不是鲁棒概念擦除的唯一步骤。在第一阶段,STEREO采用对抗训练作为漏洞识别机制以进行足够彻底的搜索。在第二阶段(即一次性鲁棒擦除阶段),STEREO引入基于锚概念的组合目标,在单个微调阶段中鲁棒地擦除目标概念,同时最小化模型实用性的退化。我们将STEREO与七种最先进的概念擦除方法进行了基准比较,证明了它对白盒和黑盒攻击的卓越鲁棒性,同时基本保留了实用性。

英文摘要:

The rapid proliferation of large-scale text-to-image diffusion (T2ID) models has raised serious concerns about their potential misuse in generating harmful content. Although numerous methods have been proposed for erasing undesired concepts from T2ID models, they often provide a false sense of security; concept-erased models (CEMs) can still be manipulated via adversarial attacks to regenerate the erased concept. While a few robust concept erasure methods based on adversarial training have emerged recently, they compromise on utility (generation quality for benign concepts) to achieve robustness and/or remain vulnerable to advanced embedding space attacks. These limitations stem from the failure of robust CEMs to thoroughly search for "blind spots" in the embedding space. To bridge this gap, we propose STEREO, a novel two-stage framework that employs adversarial training as a first step rather than the only step for robust concept erasure. In the first stage, STEREO employs adversarial training as a vulnerability identification mechanism to search thoroughly enough. In the second robustly erase once stage, STEREO introduces an anchor-concept-based compositional objective to robustly erase the target concept in a single fine-tuning stage, while minimizing the degradation of model utility. We benchmark STEREO against seven state-of-the-art concept erasure methods, demonstrating its superior robustness to both white-box and black-box attacks, while largely preserving utility.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑