DDAP：针对文本到图像扩散模型的双域反个性化
DDAP: Dual-Domain Anti-Personalization against Text-to-Image Diffusion Models
AI总结：
针对文本到图像扩散模型被滥用于生成特定个人图像的隐私风险，提出双域反个性化框架DDAP，通过交替进行空间扰动学习与频率扰动学习并引入定位模块，在保证对抗样本视觉质量的同时有效破坏个性化生成模型的输出。
摘要：
基于扩散的个性化视觉内容生成技术取得了重大突破，只需从几张参考照片中学习即可创建特定对象。然而，当这些技术被滥用来捏造针对个人的假新闻或令人不安的内容时，可能会造成相当大的社会危害。为了解决这个问题，当前的方法通过对抗性地最大化训练损失来生成对抗样本，从而破坏任何使用这些样本训练的个性化生成模型的输出。然而，现有方法未能实现有效防御并保持隐蔽性，因为它们忽略了扩散模型的内在特性。在本文中，我们提出了一种新颖的双域反个性化框架（DDAP）。具体而言，我们利用个性化生成中图像编码器固定且对扰动敏感的特性，开发了空间扰动学习（SPL）。随后，我们设计了一种频率扰动学习（FPL）方法，利用扩散模型在频域中的特性。SPL破坏生成图像的整体纹理，而FPL则专注于图像细节。通过在这两种方法之间交替，我们构建了DDAP框架，有效地利用了两个域的优势。为了进一步提高对抗样本的视觉质量，我们设计了一个定位模块来准确捕获注意力区域，同时确保攻击的有效性并避免对背景产生不必要的干扰。在面部基准上的大量实验表明，所提出的DDAP增强了对个性化生成模型的破坏，同时在对抗样本中保持了高质量，使其在实际应用中保护隐私方面更加有效。
英文摘要：
Diffusion-based personalized visual content generation technologies have achieved significant breakthroughs, allowing for the creation of specific objects by just learning from a few reference photos. However, when misused to fabricate fake news or unsettling content targeting individuals, these technologies could cause considerable societal harm. To address this problem, current methods generate adversarial samples by adversarially maximizing the training loss, thereby disrupting the output of any personalized generation model trained with these samples. However, the existing methods fail to achieve effective defense and maintain stealthiness, as they overlook the intrinsic properties of diffusion models. In this paper, we introduce a novel Dual-Domain Anti-Personalization framework (DDAP). Specifically, we have developed Spatial Perturbation Learning (SPL) by exploiting the fixed and perturbation-sensitive nature of the image encoder in personalized generation. Subsequently, we have designed a Frequency Perturbation Learning (FPL) method that utilizes the characteristics of diffusion models in the frequency domain. The SPL disrupts the overall texture of the generated images, while the FPL focuses on image details. By alternating between these two methods, we construct the DDAP framework, effectively harnessing the strengths of both domains. To further enhance the visual quality of the adversarial samples, we design a localization module to accurately capture attentive areas while ensuring the effectiveness of the attack and avoiding unnecessary disturbances in the background. Extensive experiments on facial benchmarks have shown that the proposed DDAP enhances the disruption of personalized generation models while also maintaining high quality in adversarial samples, making it more effective in protecting privacy in practical applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
