面向提示鲁棒的人脸隐私保护：对抗解耦增强框架
Towards Prompt-robust Face Privacy Protection via Adversarial Decoupling Augmentation Framework
AI总结：
针对扩散模型人脸隐私保护中忽略图像-文本融合模块及防御不稳定的问题，提出对抗解耦增强框架（ADAF），通过多级文本增强与三种损失提升防御性能，实验超越现有算法。
摘要：
去噪扩散模型在各种生成任务中展现出显著潜力。开源的大规模文本到图像模型Stable Diffusion因其能够通过少量新样本的微调生成具有个性化特征的真实艺术或人脸图像而变得流行。然而，这引发了隐私担忧，因为攻击者可以获取在线人脸图像并微调文本到图像模型进行恶意编辑，导致无端丑闻、诽谤以及扰乱受害者生活。先前的研究工作侧重于从传统训练过程中推导对抗损失，通过对抗扰动实现人脸隐私保护。然而，现有算法面临两个问题：1）它们忽略了图像-文本融合模块，这是文本到图像扩散模型的关键模块；2）它们的防御性能在面对不同攻击者提示时不稳定。在本文中，我们提出了对抗解耦增强框架（ADAF），通过针对图像-文本融合模块来解决这些问题，以增强人脸隐私保护算法的防御性能。ADAF引入了多级文本相关增强，以针对各种攻击者提示实现防御稳定性。具体而言，考虑到视觉、文本和公共单元空间，我们提出了视觉对抗损失、提示鲁棒增强和注意力解耦损失。在CelebA-HQ和VGGFace2上的大量实验表明，ADAF具有优越的性能，超越了现有算法。
英文摘要：
Denoising diffusion models have shown remarkable potential in various generation tasks. The open-source large-scale text-to-image model, Stable Diffusion, becomes prevalent as it can generate realistic artistic or facial images with personalization through fine-tuning on a limited number of new samples. However, this has raised privacy concerns as adversaries can acquire facial images online and fine-tune text-to-image models for malicious editing, leading to baseless scandals, defamation, and disruption to victims' lives. Prior research efforts have focused on deriving adversarial loss from conventional training processes for facial privacy protection through adversarial perturbations. However, existing algorithms face two issues: 1) they neglect the image-text fusion module, which is the vital module of text-to-image diffusion models, and 2) their defensive performance is unstable against different attacker prompts. In this paper, we propose the Adversarial Decoupling Augmentation Framework (ADAF), addressing these issues by targeting the image-text fusion module to enhance the defensive performance of facial privacy protection algorithms. ADAF introduces multi-level text-related augmentations for defense stability against various attacker prompts. Concretely, considering the vision, text, and common unit space, we propose Vision-Adversarial Loss, Prompt-Robust Augmentation, and Attention-Decoupling Loss. Extensive experiments on CelebA-HQ and VGGFace2 demonstrate ADAF's promising performance, surpassing existing algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
