Detecting facial forgery images and videos is an increasingly important topic in multimedia forensics. As forgery images and videos are usually compressed into different formats such as JPEG and H264 when circulating on the Internet, existing forgery-detection methods trained on uncompressed data often suffer from significant performance degradation in identifying them. To solve this problem, we propose a novel anti-compression facial forgery detection framework, which learns a compression-insensitive embedding feature space utilizing both original and compressed forgeries. Specifically, our approach consists of three ideas: (i) extracting compression-insensitive features from both uncompressed and compressed forgeries using an adversarial learning strategy; (ii) learning a robust partition by constructing a metric loss that can reduce the distance of the paired original and compressed images in the embedding space; (iii) improving the accuracy of tampered localization with an attention-transfer module. Experimental results demonstrate that, the proposed method is highly effective in handling both compressed and uncompressed facial forgery images.
University of Science and Technology of China（中国科学技术大学）
Existing face forgery detection methods usually treat face forgery detection as a binary classification problem and adopt deep convolution neural networks to learn discriminative features. The ideal discriminative features should be only related to the real/fake labels of facial images. However, we observe that the features learned by vanilla classification networks are correlated to unnecessary properties, such as forgery methods and facial identities. Such phenomenon would limit forgery detection performance especially for the generalization ability. Motivated by this, we propose a novel met…
University of Sydney（悉尼大学）；UNSW Canberra（新南威尔士大学堪培拉分校）；CSIRO Data61（澳大利亚联邦科学与工业研究组织Data61研究所）
Facial forgery detection is a crucial but extremely challenging topic, with the fast development of forgery techniques making the synthetic artefact highly indistinguishable. Prior works show that by mining both spatial and frequency information the forgery detection performance of deep learning models can be vastly improved. However, leveraging multiple types of information usually requires more than one branch in the neural network, which makes the model heavy and cumbersome. Knowledge distillation, as an important technique for efficient modelling, could be a possible remedy. We find that e…
University of Maryland College Park（马里兰大学帕克分校）；Johns Hopkins University（约翰斯·霍普金斯大学）
Manipulated videos, especially those where the identity of an individual has been modified using deep neural networks, are becoming an increasingly relevant threat in the modern day. In this paper, we seek to develop a generalizable, explainable solution to detecting these manipulated videos. To achieve this, we design a series of forgery detection systems that each focus on one individual part of the face. These parts-based detection systems, which can be combined and used together in a single architecture, meet all of our desired criteria - they generalize effectively between datasets and gi…
University of Sydney（悉尼大学）；UNSW Canberra（新南威尔士大学堪培拉分校）；CSIRO Data61（澳大利亚联邦科学与工业研究组织数据61研究所）
Forgery facial images and videos have increased the concern of digital security. It leads to the significant development of detecting forgery data recently. However, the data, especially the videos published on the Internet, are usually compressed with lossy compression algorithms such as H.264. The compressed data could significantly degrade the performance of recent detection algorithms. The existing anti-compression algorithms focus on enhancing the performance in detecting heavily compressed data but less consider the compression adaption to the data from various compression levels. We bel…
With the rapid development of generation model, AI-based face manipulation technology, which called DeepFakes, has become more and more realistic. This means of face forgery can attack any target, which poses a new threat to personal privacy and property security. Moreover, the misuse of synthetic video shows potential dangers in many areas, such as identity harassment, pornography and news rumors. Inspired by the fact that the spatial coherence and temporal consistency of physiological signal are destroyed in the generated content, we attempt to find inconsistent patterns that can distinguish…
TT-DF: A Large-Scale Diffusion-Based Dataset and Benchmark for Human Body Forgery Detection
MAIS \& NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China University of Science
SGF-CDNet: A Consistency-Discrepancy Graph Network over Semantic-Geometric Fused Nodes for Face Forgery Detection
School of Cyber Science and Technology, University of Science and Technology of China（中国科学技术大学网络空间安全学院）；Anhui Province Key Laboratory of Digital Security（安徽省数字安全重点实验室）
LAKAN: Landmark-assisted Adaptive Kolmogorov-Arnold Network for Face Forgery Detection
School of Cyber Science and Technology, University of Science and Technology of China（计算机科学与技术学院，中国科学技术大学）；Anhui Province Key Laboratory of Digital Security（安徽省数字安全重点实验室）
One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common corruptions such as compression. In this paper, we examine whether we can tackle this issue by harnessing videos of real talking faces, which contain rich information on natural facial appearance and behaviour and are readily available in large quantities online. Our method, termed RealForensics, consists of two stages. First, we exploit the natural correspondence between the visual and auditory modalities in real vi…
Technical University of Munich（慕尼黑工业大学）；University Federico II of Naples（那不勒斯费德里科二世大学）；University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）
The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the be…
A Dual-Branch CNN for Robust Detection of AI-Generated Facial Forgeries
Computer Science Department University of Southern Maine Portland, United States（计算机科学系 俄亥俄州立大学 帕斯托市 美国）；The Department of Computer Science University of Central Oklahoma Edmond, United States（计算机科学系 中央俄克拉荷马大学 埃德蒙德 美国）
École des Ponts Paristech（巴黎高科桥梁学院）；JFLI, CNRS, UMI 3527（法国国家科学研究中心JFLI联合实验室）；LIGM, UMR 8049（计算机科学、建模与应用实验室）；UPEM（巴黎东马恩-拉瓦雷大学）；National Institute of Informatics（国立情报学研究所）
This paper presents a method to automatically and efficiently detect face tampering in videos, and particularly focuses on two recent techniques used to generate hyper-realistic forged videos: Deepfake and Face2Face. Traditional image forensics techniques are usually not well suited to videos due to the compression that strongly degrades the data. Thus, this paper follows a deep learning approach and presents two networks, both with a low number of layers to focus on the mesoscopic properties of images. We evaluate those fast networks on both an existing dataset and a dataset we have constitut…
We propose Deep Distribution Transfer(DDT), a new transfer learning approach to address the problem of zero and few-shot transfer in the context of facial forgery detection. We examine how well a model (pre-)trained with one forgery creation method generalizes towards a previously unseen manipulation technique or different dataset. To facilitate this transfer, we introduce a new mixture model-based loss formulation that learns a multi-modal distribution, with modes corresponding to class categories of the underlying data of the source forgery method. Our core idea is to first pre-train an enco…
Videos are prone to tampering attacks that alter the meaning and deceive the audience. Previous video forgery detection schemes find tiny clues to locate the tampered areas. However, attackers can successfully evade supervision by destroying such clues using video compression or blurring. This paper proposes a video watermarking network for tampering localization. We jointly train a 3D-UNet-based watermark embedding network and a decoder that predicts the tampering mask. The perturbation made by watermark embedding is close to imperceptible. Considering that there is no off-the-shelf different…
MFFI: Multi-Dimensional Face Forgery Image Dataset for Real-World Scenarios
Anhui Province Key Laboratory of Digital Security（安徽省级数字安全重点实验室）；Hefei University of Technology（合肥工业大学）；Cardiff University（卡迪夫大学）；IHPC, Agency for Science, Technology and Research（科技研究局信息处理中心）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
EvolveReason: Self-Evolving Reasoning Paradigm for Explainable Deepfake Facial Image Identification
School of Software Technology, Zhejiang University（浙江大学软件技术学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Ant Group（蚂蚁集团）
StegaFFD: Privacy-Preserving Face Forgery Detection via Fine-Grained Steganographic Domain Lifting
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；Beihang University（北京航空航天大学）；Macau University of Science and Technology（澳门科技大学）；Nanyang Technological University（南洋理工大学）
This paper presents an integration of image forgery detection with image facial recognition using black propagation neural network (BPNN). We observed that facial image recognition by itself will always give a matching output or closest possible output image for every input image irrespective of the authenticity or otherwise not of the testing input image. Based on this, we are proposing the combination of the blind but powerful automation image forgery detection for entire input images for the BPNN recognition program. Hence, an input image must first be authenticated before being fed into th…
Beijing University of Posts and Telecommunications（北京邮电大学）
Although vanilla Convolutional Neural Network (CNN) based detectors can achieve satisfactory performance on fake face detection, we observe that the detectors tend to seek forgeries on a limited region of face, which reveals that the detectors is short of understanding of forgery. Therefore, we propose an attention-based data augmentation framework to guide detector refine and enlarge its attention. Specifically, our method tracks and occludes the Top-N sensitive facial regions, encouraging the detector to mine deeper into the regions ignored before for more representative forgery. Especially,…
SenseTime Research（商汤科技研究院）；University of Electronic Science and Technology of China（电子科技大学）；College of Software, Beihang University（北京航空航天大学软件学院）；Northwestern Polytechnical University（西北工业大学）
As realistic facial manipulation technologies have achieved remarkable progress, social concerns about potential malicious abuse of these technologies bring out an emerging research topic of face forgery detection. However, it is extremely challenging since recent advances are able to forge faces beyond the perception ability of human eyes, especially in compressed images and videos. We find that mining forgery patterns with the awareness of frequency could be a cure, as frequency provides a complementary viewpoint where either subtle forgery artifacts or compression errors could be well descr…
The University of Tokyo（东京大学）；National Institute of Informatics（国立信息学研究所）；Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）
Deep-learning-based technologies such as deepfakes ones have been attracting widespread attention in both society and academia, particularly ones used to synthesize forged face images. These automatic and professional-skill-free face manipulation technologies can be used to replace the face in an original image or video with any target object while maintaining the expression and demeanor. Since human faces are closely related to identity characteristics, maliciously disseminated identity manipulated videos could trigger a crisis of public trust in the media and could even have serious politica…
Learning Real Facial Concepts for Independent Deepfake Detection
School of Software, Shandong University（山东大学软件学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；Nanyang Technological University（南洋理工大学）
Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline
Xi’an Jiaotong University（西安交通大学）；Hefei University of Technology（合肥工业大学）；CSIRO（澳大利亚联邦科学与工业研究组织）；Northwestern Polytechnical University（西北工业大学）；University of Macau（澳门大学）
With the rapid development of facial manipulation techniques, face forgery detection has received considerable attention in digital media forensics due to security concerns. Most existing methods formulate face forgery detection as a classification problem and utilize binary labels or manipulated region masks as supervision. However, without considering the correlation between local regions, these global supervisions are insufficient to learn a generalized feature and prone to overfitting. To address this issue, we propose a novel perspective of face forgery detection via local relation learni…
Hanoi University of Science and Technology（河内理工大学）；Griffith University（格里菲斯大学）；Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Deakin University（迪肯大学）；The University of Queensland（昆士兰大学）
In recent years, visual forgery has reached a level of sophistication that humans cannot identify fraud, which poses a significant threat to information security. A wide range of malicious applications have emerged, such as fake news, defamation or blackmailing of celebrities, impersonation of politicians in political warfare, and the spreading of rumours to attract views. As a result, a rich body of visual forensic techniques has been proposed in an attempt to stop this dangerous trend. In this paper, we present a benchmark that provides in-depth insights into visual forgery and visual forens…
With various facial manipulation techniques arising, face forgery detection has drawn growing attention due to security concerns. Previous works always formulate face forgery detection as a classification problem based on cross-entropy loss, which emphasizes category-level differences rather than the essential discrepancies between real and fake faces, limiting model generalization in unseen domains. To address this issue, we propose a novel face forgery detection framework, named Dual Contrastive Learning (DCL), which specially constructs positive and negative paired data and performs designe…
Various facial manipulation techniques have drawn serious public concerns in morality, security, and privacy. Although existing face forgery classifiers achieve promising performance on detecting fake images, these methods are vulnerable to adversarial examples with injected imperceptible perturbations on the pixels. Meanwhile, many face forgery detectors always utilize the frequency diversity between real and fake faces as a crucial clue. In this paper, instead of injecting adversarial perturbations into the spatial domain, we propose a frequency adversarial attack method against face forgery…
Ghent University – imec（根特大学-imec）；Università degli Studi di Napoli Federico II（那不勒斯费德里科二世大学）
Manipulation tools that realistically edit images are widely available, making it easy for anyone to create and spread misinformation. In an attempt to fight fake news, forgery detection and localization methods were designed. However, existing methods struggle to accurately reveal manipulations found in images on the internet, i.e., in the wild. That is because the type of forgery is typically unknown, in addition to the tampering traces being damaged by recompression. This paper presents Comprint, a novel forgery detection and localization method based on the compression fingerprint or compr…
University of Science and Technology of China（中国科学技术大学）；Kuaishou Technology（快手科技）
Face forgery detection is raising ever-increasing interest in computer vision since facial manipulation technologies cause serious worries. Though recent works have reached sound achievements, there are still unignorable problems: a) learned features supervised by softmax loss are separable but not discriminative enough, since softmax loss does not explicitly encourage intra-class compactness and interclass separability; and b) fixed filter banks and hand-crafted features are insufficient to capture forgery patterns of frequency from diverse inputs. To compensate for such limitations, a novel …
Exploring Specular Reflection Inconsistency for Generalizable Face Forgery Detection
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；State Key Laboratory of General Artificial Intelligence, Peking University（北京大学通用人工智能国家重点实验室）；WeChat AI, Tencent Inc（腾讯公司微信AI）
MFVLR: Multi-domain Fine-grained Vision-Language Reconstruction for Generalizable Diffusion Face Forgery Detection and Localization
Faculty of Computer Science and Technology, Qilu University of Technology (Shandong Academy of Sciences)（计算机科学与技术学院，齐鲁工业大学（山东省科学院））；School of Computing, National University of Singapore（国立新加坡大学计算机学院）；Shandong Artificial Intelligence Institute, Qilu University of Technology (Shandong Academy of Sciences)（山东省人工智能研究院，齐鲁工业大学（山东省科学院））；Key Laboratory of Computer Vision and System, Ministry of Education, Tianjin University of Technology（教育部计算机视觉与系统重点实验室，天津工业大学）
Peking University（北京大学）；City University of Hong Kong（香港城市大学）
In this paper, we propose a scalable image compression scheme, including the base layer for feature representation and enhancement layer for texture representation. More specifically, the base layer is designed as the deep learning feature for analysis purpose, and it can also be converted to the fine structure with deep feature reconstruction. The enhancement layer, which serves to compress the residuals between the input image and the signals generated from the base layer, aims to faithfully reconstruct the input texture. The proposed scheme can feasibly inherit the advantages of both compre…
As the remarkable development of facial manipulation technologies is accompanied by severe security concerns, face forgery detection has become a recent research hotspot. Most existing detection methods train a binary classifier under global supervision to judge real or fake. However, advanced manipulations only perform small-scale tampering, posing challenges to comprehensively capture subtle and local forgery artifacts, especially in high compression settings and cross-dataset scenarios. To address such limitations, we propose a novel framework named Multi-modal Contrastive Classification by…
