DiffAM：用于面部隐私保护的基于扩散的对抗性妆容迁移
DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection
AI总结：
针对人脸识别系统滥用导致的面部隐私泄露问题，提出基于扩散模型的对抗妆容迁移方法DiffAM，通过卸妆模块建立妆域关系，结合CLIP妆容损失与集成攻击，生成高视觉质量、高黑盒可迁移性的保护图像，黑盒攻击成功率较SOTA提升12.98%。
摘要：
随着人脸识别（FR）系统的快速发展，由于未经授权的FR系统被滥用，社交媒体上面部图像的隐私正面临严峻挑战。已有研究利用对抗攻击技术生成对抗样本来抵御恶意FR系统，但生成的对抗样本（即受保护的面部图像）往往存在视觉质量不佳、可迁移性低的问题。本文提出一种新型面部保护方法DiffAM，该方法借助扩散模型的强大生成能力，生成从参考图像迁移对抗性妆容的高质量受保护面部图像。具体而言，我们首先引入一个卸妆模块，利用经微调的扩散模型，在CLIP空间的文本提示引导下生成无妆图像。作为妆容迁移的逆过程，卸妆无需依赖精细的文本提示，就能更轻松地建立妆容域与无妆域之间的确定性关系。随后，基于这一关系，我们引入基于CLIP的妆容损失以及集成攻击策略，共同引导对抗妆容域的方向，最终生成妆容自然、黑盒可迁移性高的受保护面部图像。大量实验表明，与现有最优方法相比，DiffAM在黑盒设置下实现了更高的视觉质量和攻击成功率，攻击成功率提升了12.98%。代码将发布于https://github.com/HansSunY/DiffAM。
英文摘要：
With the rapid development of face recognition (FR) systems, the privacy of face images on social media is facing severe challenges due to the abuse of unauthorized FR systems. Some studies utilize adversarial attack techniques to defend against malicious FR systems by generating adversarial examples. However, the generated adversarial examples, i.e., the protected face images, tend to suffer from subpar visual quality and low transferability. In this paper, we propose a novel face protection approach, dubbed DiffAM, which leverages the powerful generative ability of diffusion models to generate high-quality protected face images with adversarial makeup transferred from reference images. To be specific, we first introduce a makeup removal module to generate non-makeup images utilizing a fine-tuned diffusion model with guidance of textual prompts in CLIP space. As the inverse process of makeup transfer, makeup removal can make it easier to establish the deterministic relationship between makeup domain and non-makeup domain regardless of elaborate text prompts. Then, with this relationship, a CLIP-based makeup loss along with an ensemble attack strategy is introduced to jointly guide the direction of adversarial makeup domain, achieving the generation of protected face images with natural-looking makeup and high black-box transferability. Extensive experiments demonstrate that DiffAM achieves higher visual quality and attack success rates with a gain of 12.98% under black-box setting compared with the state of the arts. The code will be available at https://github.com/HansSunY/DiffAM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
