FoolSDEdit：欺骗性地将编辑导向目标属性感知分布
FoolSDEdit: Deceptively Steering Your Edits Towards Targeted Attribute-aware Distribution
AI总结：
针对SDEdit可能生成违背用户意图图像的安全漏洞，提出TAGA攻击方法，通过联合优化曝光与运动模糊扰动，迫使SDEdit生成指定属性分布，实验验证其有效性。
摘要：
基于扩散模型的SDEdit等引导图像合成方法，在根据用户输入（如笔画画）创建逼真图像方面表现出色。然而，现有工作主要关注图像质量，往往忽略了一个关键点：扩散模型表示的是数据分布，而非单个图像。这引入了较低但关键的概率，即生成与用户意图相矛盾的图像，引发伦理问题。例如，用户输入具有女性特征的笔画画时，SDEdit可能以一定概率生成男性面孔。为暴露这一潜在漏洞，我们旨在构建一种对抗性攻击，迫使SDEdit生成与指定属性（如女性）对齐的特定数据分布，同时不改变输入的属性特征。我们提出了目标属性生成攻击（TAGA），使用属性感知的目标函数，并优化添加到输入笔画画上的对抗性噪声。实证研究表明，传统对抗性噪声难以应对TAGA，而自然扰动（如曝光和运动模糊）则容易改变生成图像的属性。为实施有效攻击，我们引入了FoolSDEdit：我们设计了一种联合对抗性曝光和模糊攻击，将曝光和运动模糊添加到笔画画中并联合优化它们。我们将各种扰动的执行策略优化问题构建为网络架构搜索问题。我们创建了SuperPert，一个表示不同扰动多样化执行策略的图。训练后，我们获得了针对SDEdit的有效TAGA优化执行策略。在两个数据集上的全面实验表明，我们的方法能迫使SDEdit生成目标属性感知的数据分布，显著优于基线方法。
英文摘要：
Guided image synthesis methods, like SDEdit based on the diffusion model, excel at creating realistic images from user inputs such as stroke paintings. However, existing efforts mainly focus on image quality, often overlooking a key point: the diffusion model represents a data distribution, not individual images. This introduces a low but critical chance of generating images that contradict user intentions, raising ethical concerns. For example, a user inputting a stroke painting with female characteristics might, with some probability, get male faces from SDEdit. To expose this potential vulnerability, we aim to build an adversarial attack forcing SDEdit to generate a specific data distribution aligned with a specified attribute (e.g., female), without changing the input's attribute characteristics. We propose the Targeted Attribute Generative Attack (TAGA), using an attribute-aware objective function and optimizing the adversarial noise added to the input stroke painting. Empirical studies reveal that traditional adversarial noise struggles with TAGA, while natural perturbations like exposure and motion blur easily alter generated images' attributes. To execute effective attacks, we introduce FoolSDEdit: We design a joint adversarial exposure and blur attack, adding exposure and motion blur to the stroke painting and optimizing them together. We optimize the execution strategy of various perturbations, framing it as a network architecture search problem. We create the SuperPert, a graph representing diverse execution strategies for different perturbations. After training, we obtain the optimized execution strategy for effective TAGA against SDEdit. Comprehensive experiments on two datasets show our method compelling SDEdit to generate a targeted attribute-aware data distribution, significantly outperforming baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
