The detection of malicious deepfakes is a constantly evolving problem that requires continuous monitoring of detectors to ensure they can detect image manipulations generated by the latest emerging models. In this paper, we investigate the vulnerability of single-image deepfake detectors to black-box attacks created by the newest generation of generative methods, namely Denoising Diffusion Models (DDMs). Our experiments are run on FaceForensics++, a widely used deepfake benchmark consisting of manipulated images generated with various techniques for face identity swapping and face reenactment. Attacks are crafted through guided reconstruction of existing deepfakes with a proposed DDM approach for face restoration. Our findings indicate that employing just a single denoising diffusion step in the reconstruction process of a deepfake can significantly reduce the likelihood of detection, all without introducing any perceptible image modifications. While training detectors using attack examples demonstrated some effectiveness, it was observed that discriminators trained on fully diffusion-based deepfakes exhibited limited generalizability when presented with our attacks.
Recent advances in video manipulation techniques have made the generation of fake videos more accessible than ever before. Manipulated videos can fuel disinformation and reduce trust in media. Therefore detection of fake videos has garnered immense interest in academia and industry. Recently developed Deepfake detection methods rely on deep neural networks (DNNs) to distinguish AI-generated fake videos from real videos. In this work, we demonstrate that it is possible to bypass such detectors by adversarially modifying fake videos synthesized using existing Deepfake generation methods. We furt…
Inconsistency-aware Multimodal Schrödinger Bridge for Deepfake Localization
Department of Computer Science and Techonology, Huaqiao University（华侨大学计算机科学与技术系）；Xiamen Key Laboratory of Computer Vision and Pattern Recognition, Huaqiao University（厦门计算机视觉与模式识别重点实验室）；Tongji University（同济大学）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）
Vulnerability-Aware Spatio-Temporal Learning for Generalizable Deepfake Video Detection
CVI 2 , SnT, University of Luxembourg（CVI 2 ，SnT，卢森堡大学）；Cristal Laboratory, National School of Computer Sciences, University of Manouba（Cristal实验室，国家计算机科学学校，曼努巴大学）
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院（EPFL））
Deepfakes are becoming increasingly popular in both good faith applications such as in entertainment and maliciously intended manipulations such as in image and video forgery. Primarily motivated by the latter, a large number of deepfake detectors have been proposed recently in order to identify such content. While the performance of such detectors still need further improvements, they are often assessed in simple if not trivial scenarios. In particular, the impact of benign processing operations such as transcoding, denoising, resizing and enhancement are not sufficiently studied. This paper …
Towards Robust Defense against Customization via Protective Perturbation Resistant to Diffusion-based Purification
MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences（自动化研究所、中国科学院）；School of Artificial Intelligence, University of Chinese Academy of Sciences（人工智能学院、中国科学院大学）
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；University of Maryland, College Park（马里兰大学帕克分校）；University of Mannheim（曼海姆大学）
The rapid advances in deep generative models over the past years have led to highly {realistic media, known as deepfakes,} that are commonly indistinguishable from real to human eyes. These advances make assessing the authenticity of visual data increasingly difficult and pose a misinformation threat to the trustworthiness of visual content in general. Although recent work has shown strong detection accuracy of such deepfakes, the success largely relies on identifying frequency artifacts in the generated images, which will not yield a sustainable detection approach as generative models continu…
Exposing and Mitigating Temporal Attack in Deepfake Video Detection
Peking University（北京大学）；New York University（纽约大学）；Huzhou University（湖州大学）；Institute of Science Tokyo（东京科学研究院）；Guangdong Institute of Intelligence Science and Technology（广东省智能科学与技术研究院）
Pennsylvania State University（宾夕法尼亚州立大学）；Zhejiang University（浙江大学）；Shandong University（山东大学）
Facial Liveness Verification (FLV) is widely used for identity authentication in many security-sensitive domains and offered as Platform-as-a-Service (PaaS) by leading cloud vendors. Yet, with the rapid advances in synthetic media techniques (e.g., deepfake), the security of FLV is facing unprecedented challenges, about which little is known thus far. To bridge this gap, in this paper, we conduct the first systematic study on the security of FLV in real-world settings. Specifically, we present LiveBugger, a new deepfake-powered attack framework that enables customizable, automated security eva…
Technion - Israel Institute of Technology（以色列理工学院）
Deep Neural Networks (DNNs) are highly sensitive to imperceptible malicious perturbations, known as adversarial attacks. Following the discovery of this vulnerability in real-world imaging and vision applications, the associated safety concerns have attracted vast research attention, and many defense techniques have been developed. Most of these defense methods rely on adversarial training (AT) -- training the classification network on images perturbed according to a specific threat model, which defines the magnitude of the allowed modification. Although AT leads to promising results, training…
MIGA: Mutual Information-Guided Attack on Denoising Models for Semantic Manipulation
SIGS, Tsinghua University（清华大学 SIGS）；Southern University of Science and Technology（南方科技大学）；Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)（广东省人工智能与数字经济实验室（深圳））
Yonsei University（延世大学）；University of California, Los Angeles（加州大学洛杉矶分校）
Recently, the vulnerability of deep image classification models to adversarial attacks has been investigated. However, such an issue has not been thoroughly studied for image-to-image tasks that take an input image and generate an output image (e.g., colorization, denoising, deblurring, etc.) This paper presents comprehensive investigations into the vulnerability of deep image-to-image models to adversarial attacks. For five popular image-to-image tasks, 16 deep models are analyzed from various standpoints such as output quality degradation due to attacks, transferability of adversarial exampl…
It is increasingly easy to automatically swap faces in images and video or morph two faces into one using generative adversarial networks (GANs). The high quality of the resulted deep-morph raises the question of how vulnerable the current face recognition systems are to such fake images and videos. It also calls for automated ways to detect these GAN-generated faces. In this paper, we present the publicly available dataset of the Deepfake videos with faces morphed with a GAN-based algorithm. To generate these videos, we used open source software based on GANs, and we emphasize that training a…
Deep learning models have shown their vulnerability when dealing with adversarial attacks. Existing attacks almost perform on low-level instances, such as pixels and super-pixels, and rarely exploit semantic clues. For face recognition attacks, existing methods typically generate the l_p-norm perturbations on pixels, however, resulting in low attack transferability and high vulnerability to denoising defense models. In this work, instead of performing perturbations on the low-level pixels, we propose to generate attacks through perturbing on the high-level semantics to improve attack transfera…
Nanyang Technological University（南洋理工大学）；Alibaba Group（阿里巴巴集团）；Tianjin University（天津大学）；State Administration of Cultural Heritage（国家文物局）
Image denoising can remove natural noise that widely exists in images captured by multimedia devices due to low-quality imaging sensors, unstable image transmission processes, or low light conditions. Recent works also find that image denoising benefits the high-level vision tasks, e.g., image classification. In this work, we try to challenge this common sense and explore a totally new problem, i.e., whether the image denoising can be given the capability of fooling the state-of-the-art deep neural networks (DNNs) while enhancing the image quality. To this end, we initiate the very first attem…
Benchmarking Unified Face Attack Detection via Hierarchical Prompt Tuning
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences (CASIA)（多模态人工智能系统国家重点实验室（MAIS），自动化研究所，中国科学院（CASIA））；Department of Computer Science, College of Computing, City University of Hong Kong（计算机科学系，计算学院，香港城市大学）；School of Computer Science and Engineering, Faculty of Innovation Engineering, Macau University of Science and Technology（计算机科学与工程学院，创新工程学院，澳门科学理工学院）；Department of Computer Science and Engineering, Michigan State University（计算机科学与工程系，密歇根州立大学）；School of Cyber Science and Technology, University of Science and Technology of China（网络科学与技术学院，中国科学技术大学）；Ant Group（蚂蚁集团）；Imperial College London（伦敦帝国理工学院）
Nanyang Technological University（南洋理工大学）；East China Normal University（华东师范大学）；Agency for Science, Technology and Research（新加坡科技研究局）；Meta AI；Shanghai Industrial Control Safety Innovation Tech. Co., Ltd（上海工控安全创新科技有限公司）；Zhejiang Sci-Tech University（浙江理工大学）
The Brittleness of AI-Generated Image Watermarking Techniques: Examining Their Robustness Against Visual Paraphrasing Attacks
NIT Silchar（西尔恰尔国家理工学院）；IIIT Delhi（德里英迪拉·甘地信息技术学院）；BITS Pilani Hyderabad（博拉理工学院海得拉巴校区）；Meta AI；Stanford University（斯坦福大学）；Amazon GenAI（亚马逊生成式人工智能）；AI Institute, University of South Carolina（南卡罗来纳大学人工智能研究院）
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；School of Computer Science and Cyber Security, Chengdu University of Technology（成都理工大学计算机与网络安全学院）；City University of Hong Kong（香港城市大学）；School of Computer Science and Engineering, Nanyang Technological University（南洋理工大学计算机科学与工程学院）；Faculty of Computer Science, University of New Brunswick（新不伦瑞克大学计算机科学学院）
