Various facial manipulation techniques have drawn serious public concerns in morality, security, and privacy. Although existing face forgery classifiers achieve promising performance on detecting fake images, these methods are vulnerable to adversarial examples with injected imperceptible perturbations on the pixels. Meanwhile, many face forgery detectors always utilize the frequency diversity between real and fake faces as a crucial clue. In this paper, instead of injecting adversarial perturbations into the spatial domain, we propose a frequency adversarial attack method against face forgery detectors. Concretely, we apply discrete cosine transform (DCT) on the input images and introduce a fusion module to capture the salient region of adversary in the frequency domain. Compared with existing adversarial attacks (e.g. FGSM, PGD) in the spatial domain, our method is more imperceptible to human observers and does not degrade the visual quality of the original images. Moreover, inspired by the idea of meta-learning, we also propose a hybrid adversarial attack that performs attacks in both the spatial and frequency domains. Extensive experiments indicate that the proposed method fools not only the spatial-based detectors but also the state-of-the-art frequency-based detectors effectively. In addition, the proposed frequency attack enhances the transferability across face forgery detectors as black-box attacks.
Active Adversarial Noise Suppression for Image Forgery Localization
College of Electronic and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）；Shenzhen Key Laboratory of Media Security（深圳媒体安全重点实验室）；Guangdong Laboratory of Machine Perception and Intelligent Computing, Faculty of Engineering, Shenzhen MSU-BIT University（广东机器感知与智能计算实验室，深圳MSU-BIT大学工程学院）
Counterfactual Explanations for Face Forgery Detection via Adversarial Removal of Artifacts
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Nanjing University of Information Science and Technology（南京信息工程大学）
ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack
Shenzhen Key Laboratory of Media Security, Faculty of Electronic and Information Engineering, Shenzhen University, China（深圳媒体安全重点实验室，电子与信息工程学院，深圳大学，中国）；Rapid-Rich Object Search (ROSE) Lab, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（快速富对象搜索（ROSE）实验室，电气电子工程学院，南洋理工大学，新加坡）；Guangdong Laboratory of Machine Perception and Intelligent Computing, Faculty of Engineering, Shenzhen MSU-BIT University, China（广东机器感知与智能计算实验室，工程学院，深圳MSU-BIT大学，中国）
With the ubiquitous diffusion of social networks, images are becoming a dominant and powerful communication channel. Not surprisingly, they are also increasingly subject to manipulations aimed at distorting information and spreading fake news. In recent years, the scientific community has devoted major efforts to contrast this menace, and many image forgery detectors have been proposed. Currently, due to the success of deep learning in many multimedia processing tasks, there is high interest towards CNN-based detectors, and early results are already very promising. Recent studies in computer v…
Exploring the Adversarial Robustness of Face Forgery Detection with Decision-based Black-box Attacks
Shanghai Engineering Research Center of AI & Robotics, Academy for Engineering & Technology, Fudan University, Shanghai 200433, China（上海人工智能与机器人工程研究中心，工程与技术学院，复旦大学，上海）；Engineering Research Center of Robotics, Ministry of Education, Academy for Engineering & Technology, Fudan University, Shanghai 200433, China（机器人工程研究中心，教育部，工程与技术学院，复旦大学，上海）；Youtu Lab, Tencent, Shanghai 200000, China（优衣库实验室，腾讯，上海）；vivo Mobile Communication Co., Ltd, Shanghai 200120, China（vivo移动通信有限公司，上海）；Shanghai Key Lab of Intelligent Information Processing, School of Computer Science, Fudan University, Shanghai 200433, China（上海智能信息处理重点实验室，计算机科学学院，复旦大学，上海）
The University of Adelaide（阿德莱德大学）；Tencent AI Lab（腾讯人工智能实验室）
Recent studies in deepfake detection have yielded promising results when the training and testing face forgeries are from the same dataset. However, the problem remains challenging when one tries to generalize the detector to forgeries created by unseen methods in the training dataset. This work addresses the generalizable deepfake detection from a simple principle: a generalizable representation should be sensitive to diverse types of forgeries. Following this principle, we propose to enrich the "diversity" of forgeries by synthesizing augmented forgeries with a pool of forgery configurations…
University of Science and Technology of China（中国科学技术大学）
Existing face forgery detection methods usually treat face forgery detection as a binary classification problem and adopt deep convolution neural networks to learn discriminative features. The ideal discriminative features should be only related to the real/fake labels of facial images. However, we observe that the features learned by vanilla classification networks are correlated to unnecessary properties, such as forgery methods and facial identities. Such phenomenon would limit forgery detection performance especially for the generalization ability. Motivated by this, we propose a novel met…
ED$^4$: Explicit Data-level Debiasing for Deepfake Detection
School of computer science, Wuhan University（武汉大学计算机学院）；WeChat, Tencent Inc.（腾讯公司）；School of electronic and computer engineering, Peking University Shenzhen Graduate School（北京大学深圳研究生院电子与计算机工程学院）
ByteDance AI Lab（字节跳动人工智能实验室）；Harbin Institute of Technology（哈尔滨工业大学）
With the progress in AI-based facial forgery (i.e., deepfake), people are increasingly concerned about its abuse. Albeit effort has been made for training classification (also known as deepfake detection) models to recognize such forgeries, existing models suffer from poor generalization to unseen forgery technologies and high sensitivity to changes in image/video quality. In this paper, we advocate adversarial training for improving the generalization ability to both unseen facial forgeries and unseen image/video qualities. We believe training with samples that are adversarially crafted to at…
Donghua University（东华大学）；National Institute of Informatics（国立信息学研究所）；SOKENDAI (The Graduate University for Advanced Studies)（综合研究大学院大学（高等研究生院大学））
Deep neural networks (DNNs) have been used in digital forensics to identify fake facial images. We investigated several DNN-based forgery forensics models (FFMs) to examine whether they are secure against adversarial attacks. We experimentally demonstrated the existence of individual adversarial perturbations (IAPs) and universal adversarial perturbations (UAPs) that can lead a well-performed FFM to misbehave. Based on iterative procedure, gradient information is used to generate two kinds of IAPs that can be used to fabricate classification and segmentation outputs. In contrast, UAPs are gene…
The transferability and robustness of adversarial examples are two practical yet important properties for black-box adversarial attacks. In this paper, we explore effective mechanisms to boost both of them from the perspective of network hierarchy, where a typical network can be hierarchically divided into output stage, intermediate stage and input stage. Since over-specialization of source model, we can hardly improve the transferability and robustness of the adversarial perturbations in the output stage. Therefore, we focus on the intermediate and input stages in this paper and propose a tra…
University of Chinese Academy of Sciences（中国科学院大学）；Center for Research on Intelligent Perception and Computing, CASIA（中国科学院自动化研究所智能感知与计算研究中心）
Images synthesized by powerful generative adversarial network (GAN) based methods have drawn moral and privacy concerns. Although image forensic models have reached great performance in detecting fake images from real ones, these models can be easily fooled with a simple adversarial attack. But, the noise adding adversarial samples are also arousing suspicion. In this paper, instead of adding adversarial noise, we optimally search adversarial points on face manifold to generate anti-forensic fake face images. We iteratively do a gradient-descent with each small step in the latent space of a ge…
Digital watermarking has been widely used to protect the copyright and integrity of multimedia data. Previous studies mainly focus on designing watermarking techniques that are robust to attacks of destroying the embedded watermarks. However, the emerging deep learning based image generation technology raises new open issues that whether it is possible to generate fake watermarked images for circumvention. In this paper, we make the first attempt to develop digital image watermark fakers by using generative adversarial learning. Suppose that a set of paired images of original and watermarked i…
Beijing University of Posts and Telecommunications（北京邮电大学）；SenseTime Research（商汤科技研究院）
Convolutional neural network based face forgery detection methods have achieved remarkable results during training, but struggled to maintain comparable performance during testing. We observe that the detector is prone to focus more on content information than artifact traces, suggesting that the detector is sensitive to the intrinsic bias of the dataset, which leads to severe overfitting. Motivated by this key observation, we design an easily embeddable disentanglement framework for content information removal, and further propose a Content Consistency Constraint (C2C) and a Global Representa…
RSC-SNN: Exploring the Trade-off Between Adversarial Robustness and Accuracy in Spiking Neural Networks via Randomized Smoothing Coding
Chongqing University（重庆大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Xi’an Jiaotong University（西安交通大学）；Software Security Technology Company Ltd（软件安全技术有限公司）
Revisiting and Exploring Efficient Fast Adversarial Training via LAW: Lipschitz Regularization and Auto Weight Averaging
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Alibaba Group（阿里巴巴集团）；University of Oxford（牛津大学）；Sun Yat-sen University（中山大学）
Xiamen University（厦门大学）；Microsoft Research Asia（微软亚洲研究院）
Although current face manipulation techniques achieve impressive performance regarding quality and controllability, they are struggling to generate temporal coherent face videos. In this work, we explore to take full advantage of the temporal coherence for video face forgery detection. To achieve this, we propose a novel end-to-end framework, which consists of two major stages. The first stage is a fully temporal convolution network (FTCN). The key insight of FTCN is to reduce the spatial convolution kernel size to 1, while maintaining the temporal convolution kernel size unchanged. We surpris…
University of Technology Sydney（悉尼科技大学）；Swinburne University of Technology（斯威本科技大学）；City University of Macau（澳门城市大学）
DeepFakes are raising significant social concerns. Although various DeepFake detectors have been developed as forensic countermeasures, these detectors are still vulnerable to attacks. Recently, a few attacks, principally adversarial attacks, have succeeded in cloaking DeepFake images to evade detection. However, these attacks have typical detector-specific designs, which require prior knowledge about the detector, leading to poor transferability. Moreover, these attacks only consider simple security scenarios. Less is known about how effective they are in high-level scenarios where either the…
This work conducts the first analysis on the robustness against adversarial attacks on self-supervised Vision Transformers trained using DINO. First, we evaluate whether features learned through self-supervision are more robust to adversarial attacks than those emerging from supervised learning. Then, we present properties arising for attacks in the latent space. Finally, we evaluate whether three well-known defense strategies can increase adversarial robustness in downstream tasks by only fine-tuning the classification head to provide robustness even in view of limited compute resources. Thes…
Exploring Specular Reflection Inconsistency for Generalizable Face Forgery Detection
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）；WeChat AI, Tencent Inc（腾讯公司微信AI）
Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation
Shanghai Engineering Research Center of AI & Robotics, Academy for Engineering & Technology, Fudan University（复旦大学工程与应用技术研究院，上海人工智能与机器人工程技术研究中心）；Engineering Research Center of AI & Robotics, Ministry of Education, Academy for Engineering & Technology, Fudan University（复旦大学工程与应用技术研究院，教育部人工智能与机器人工程技术研究中心）
Adversarially Robust AI-Generated Image Detection for Free: An Information Theoretic Perspective
Hefei University of Technology（合肥工业大学）；University College London（伦敦大学学院）；Xi’an Jiaotong University（西安交通大学）；Xinjiang University（新疆大学）；University of Science and Technology of China（中国科学技术大学）
Shenzhen Institute of Artificial Intelligence and Robotics for Society（深圳市人工智能与机器人研究院）；Shenzhen University（深圳大学）
Image editing techniques enable people to modify the content of an image without leaving visual traces and thus may cause serious security risks. Hence the detection and localization of these forgeries become quite necessary and challenging. Furthermore, unlike other tasks with extensive data, there is usually a lack of annotated forged images for training due to annotation difficulties. In this paper, we propose a self-adversarial training strategy and a reliable coarse-to-fine network that utilizes a self-attention mechanism to localize forged regions in forgery images. The self-attention mo…
DeepFake face swapping presents a significant threat to online security and social media, which can replace the source face in an arbitrary photo/video with the target face of an entirely different person. In order to prevent this fraud, some researchers have begun to study the adversarial methods against DeepFake or face manipulation. However, existing works focus on the white-box setting or the black-box setting driven by abundant queries, which severely limits the practical application of these methods. To tackle this problem, we introduce a practical adversarial attack that does not requir…
Deep neural networks are learning models having achieved state of the art performance in many fields like prediction, computer vision, language processing and so on. However, it has been shown that certain inputs exist which would not trick a human normally, but may mislead the model completely. These inputs are known as adversarial inputs. These inputs pose a high security threat when such models are used in real world applications. In this work, we have analyzed the resistance of three different classes of fully connected dense networks against the rarely tested non-gradient based adversaria…
School of Computing and Information Systems, The University of Melbourne（墨尔本大学计算与信息系统学院）；Key Lab. of Machine Perception, School of Artificial Intelligence, Peking University（北京大学人工智能学院机器感知重点实验室）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；University of California, Los Angeles（加州大学洛杉矶分校）；School of Computer Science, Fudan University（复旦大学计算机科学学院）
Deep neural networks (DNNs) are known to be vulnerable to adversarial attacks. A range of defense methods have been proposed to train adversarially robust DNNs, among which adversarial training has demonstrated promising results. However, despite preliminary understandings developed for adversarial training, it is still not clear, from the architectural perspective, what configurations can lead to more robust DNNs. In this paper, we address this gap via a comprehensive investigation on the impact of network width and depth on the robustness of adversarially trained DNNs. Specifically, we make …
Deep neural networks are vulnerable to adversarial attacks. White-box adversarial attacks can fool neural networks with small adversarial perturbations, especially for large size images. However, keeping successful adversarial perturbations imperceptible is especially challenging for transfer-based black-box adversarial attacks. Often such adversarial examples can be easily spotted due to their unpleasantly poor visual qualities, which compromises the threat of adversarial attacks in practice. In this study, to improve the image quality of black-box adversarial examples perceptually, we propos…
Embry-Riddle Aeronautical University（安柏瑞德航空大学）；University of Colorado Boulder（科罗拉多大学博尔德分校）；University of Tennessee at Martin（田纳西大学马丁分校）；Bowling Green State University（鲍灵格林州立大学）；Purdue University（普渡大学）；Trier University of Applied Sciences（特里尔应用技术大学）
Deep Learning (DL) is being applied in various domains, especially in safety-critical applications such as autonomous driving. Consequently, it is of great significance to ensure the robustness of these methods and thus counteract uncertain behaviors caused by adversarial attacks. In this paper, we use gradient heatmaps to analyze the response characteristics of the VGG-16 model when the input images are mixed with adversarial noise and statistically similar Gaussian random noise. In particular, we compare the network response layer by layer to determine where errors occurred. Several interest…
Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods
Lomonosov Moscow State University（莫斯科罗蒙诺索夫国立大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院信息学与管理研究中心可信人工智能研究中心）；MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能研究所）；Innopolis University（Innopolis大学）
School of Computer Science, Wuhan University（武汉大学计算机学院）
Detecting facial forgery images and videos is an increasingly important topic in multimedia forensics. As forgery images and videos are usually compressed into different formats such as JPEG and H264 when circulating on the Internet, existing forgery-detection methods trained on uncompressed data often suffer from significant performance degradation in identifying them. To solve this problem, we propose a novel anti-compression facial forgery detection framework, which learns a compression-insensitive embedding feature space utilizing both original and compressed forgeries. Specifically, our a…
DeepfakeBench-MM: A Comprehensive Benchmark for Multimodal Deepfake Detection
The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University at Buffalo, State University of New York（纽约州立大学布法罗分校）；Nanjing University of Posts and Telecommunications（南京邮电大学）
Exploring Adversarial Robustness and Safety Alignment in Multilingual Multi-Modal Large Language Models
Mohamed Bin Zayed University of AI, UAE（穆罕默德·本·扎耶德人工智能大学，阿联酋）；Khalifa University, UAE（卡比拉大学，阿联酋）；Australian National University, Australia（澳大利亚国立大学，澳大利亚）
Recent advances in autoencoders and generative models have given rise to effective video forgery methods, used for generating so-called "deepfakes". Mitigation research is mostly focused on post-factum deepfake detection and not on prevention. We complement these efforts by introducing a novel class of adversarial attacks---training-resistant attacks---which can disrupt face-swapping autoencoders whether or not its adversarial images have been included in the training set of said autoencoders. We propose the Oscillating GAN (OGAN) attack, a novel attack optimized to be training-resistant, whic…
DeepForgeSeal: Latent Space-Driven Semi-Fragile Watermarking for Deepfake Detection Using Adversarial Reinforcement Learning
The Signal Processing, Artificial Intelligence and Vision Technologies (SAIVT), Queensland University of Technology, Australia（信号处理、人工智能与视觉技术研究所（SAIVT），昆士兰理工大学）
University of Macau（澳门大学）；Macau University of Science and Technology（澳门科技大学）
With the continuous research on Deepfake forensics, recent studies have attempted to provide the fine-grained localization of forgeries, in addition to the coarse classification at the video-level. However, the detection and localization performance of existing Deepfake forensic methods still have plenty of room for further improvement. In this work, we propose a Spatial-Temporal Deepfake Detection and Localization (ST-DDL) network that simultaneously explores spatial and temporal features for detecting and localizing forged regions. Specifically, we design a new Anchor-Mesh Motion (AMM) algor…
