面向定制化扩散模型的提示无关对抗扰动
Prompt-Agnostic Adversarial Perturbation for Customized Diffusion Models
AI总结：
针对定制化扩散模型的提示特定对抗保护方法适配性不足的问题，提出基于拉普拉斯近似建模提示分布、最大化扰动期望的提示无关对抗扰动方法，提升了防御稳定性与泛化性。
摘要：
扩散模型已彻底革新了定制化文本到图像生成技术，能够借助文本描述从个人数据高效合成照片。然而，这些进展也带来了隐私泄露、艺术品未经授权复制等风险。以往研究主要围绕使用提示特定方法生成对抗样本以保护个人图像，但现有方法的效果受限于对不同提示的适配性不足。本文提出一种面向定制化扩散模型的提示无关对抗扰动（Prompt-Agnostic Adversarial Perturbation，PAP）方法。PAP首先利用拉普拉斯近似（Laplace Approximation）对提示分布建模，随后通过最大化基于该建模分布的扰动期望，生成与提示无关的扰动。该方法可有效应对提示无关攻击，提升防御稳定性。在人脸隐私和艺术风格保护场景下开展的大量实验表明，与现有技术相比，PAP具备更优异的泛化能力。我们的项目页面位于：https://github.com/vancyland/Prompt-Agnostic-Adversarial-Perturbation-for-Customized-Diffusion-Models.github.io。
英文摘要：
Diffusion models have revolutionized customized text-to-image generation, allowing for efficient synthesis of photos from personal data with textual descriptions. However, these advancements bring forth risks including privacy breaches and unauthorized replication of artworks. Previous researches primarily center around using prompt-specific methods to generate adversarial examples to protect personal images, yet the effectiveness of existing methods is hindered by constrained adaptability to different prompts. In this paper, we introduce a Prompt-Agnostic Adversarial Perturbation (PAP) method for customized diffusion models. PAP first models the prompt distribution using a Laplace Approximation, and then produces prompt-agnostic perturbations by maximizing a disturbance expectation based on the modeled distribution. This approach effectively tackles the prompt-agnostic attacks, leading to improved defense stability. Extensive experiments in face privacy and artistic style protection, demonstrate the superior generalization of PAP in comparison to existing techniques. Our project page is available at https://github.com/vancyland/Prompt-Agnostic-Adversarial-Perturbation-for-Customized-Diffusion-Models.github.io.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
