Forgery facial images and videos have increased the concern of digital security. It leads to the significant development of detecting forgery data recently. However, the data, especially the videos published on the Internet, are usually compressed with lossy compression algorithms such as H.264. The compressed data could significantly degrade the performance of recent detection algorithms. The existing anti-compression algorithms focus on enhancing the performance in detecting heavily compressed data but less consider the compression adaption to the data from various compression levels. We believe creating a forgery detection model that can handle the data compressed with unknown levels is important. To enhance the performance for such models, we consider the weak compressed and strong compressed data as two views of the original data and they should have similar representation and relationships with other samples. We propose a novel anti-compression forgery detection framework by maintaining closer relations within data under different compression levels. Specifically, the algorithm measures the pair-wise similarity within data as the relations, and forcing the relations of weak and strong compressed data close to each other, thus improving the discriminate power for detecting strong compressed data. To achieve a better strong compressed data relation guided by the less compressed one, we apply video level contrastive learning for weak compressed data, which forces the model to produce similar representations within the same video and far from the negative samples. The experiment results show that the proposed algorithm could boost performance for strong compressed data while improving the accuracy rate when detecting the clean data.
School of Computer Science, Wuhan University（武汉大学计算机学院）
Detecting facial forgery images and videos is an increasingly important topic in multimedia forensics. As forgery images and videos are usually compressed into different formats such as JPEG and H264 when circulating on the Internet, existing forgery-detection methods trained on uncompressed data often suffer from significant performance degradation in identifying them. To solve this problem, we propose a novel anti-compression facial forgery detection framework, which learns a compression-insensitive embedding feature space utilizing both original and compressed forgeries. Specifically, our a…
University of Science and Technology of China（中国科学技术大学）
Existing face forgery detection methods usually treat face forgery detection as a binary classification problem and adopt deep convolution neural networks to learn discriminative features. The ideal discriminative features should be only related to the real/fake labels of facial images. However, we observe that the features learned by vanilla classification networks are correlated to unnecessary properties, such as forgery methods and facial identities. Such phenomenon would limit forgery detection performance especially for the generalization ability. Motivated by this, we propose a novel met…
University of Sydney（悉尼大学）；UNSW Canberra（新南威尔士大学堪培拉分校）；CSIRO Data61（澳大利亚联邦科学与工业研究组织Data61研究所）
Facial forgery detection is a crucial but extremely challenging topic, with the fast development of forgery techniques making the synthetic artefact highly indistinguishable. Prior works show that by mining both spatial and frequency information the forgery detection performance of deep learning models can be vastly improved. However, leveraging multiple types of information usually requires more than one branch in the neural network, which makes the model heavy and cumbersome. Knowledge distillation, as an important technique for efficient modelling, could be a possible remedy. We find that e…
Center for Computational & Data Sciences, Independent University, Bangladesh（孟加拉国独立大学计算与数据科学中心）；Sungkyunkwan University（成均馆大学）
Conventional forgery localizing methods usually rely on different forgery footprints such as JPEG artifacts, edge inconsistency, camera noise, etc., with cross-entropy loss to locate manipulated regions. However, these methods have the disadvantage of over-fitting and focusing on only a few specific forgery footprints. On the other hand, real-life manipulated images are generated via a wide variety of forgery operations and thus, leave behind a wide variety of forgery footprints. Therefore, we need a more general approach for image forgery localization that can work well on a variety of forger…
With various facial manipulation techniques arising, face forgery detection has drawn growing attention due to security concerns. Previous works always formulate face forgery detection as a classification problem based on cross-entropy loss, which emphasizes category-level differences rather than the essential discrepancies between real and fake faces, limiting model generalization in unseen domains. To address this issue, we propose a novel face forgery detection framework, named Dual Contrastive Learning (DCL), which specially constructs positive and negative paired data and performs designe…
University of Maryland College Park（马里兰大学帕克分校）；Johns Hopkins University（约翰斯·霍普金斯大学）
Manipulated videos, especially those where the identity of an individual has been modified using deep neural networks, are becoming an increasingly relevant threat in the modern day. In this paper, we seek to develop a generalizable, explainable solution to detecting these manipulated videos. To achieve this, we design a series of forgery detection systems that each focus on one individual part of the face. These parts-based detection systems, which can be combined and used together in a single architecture, meet all of our desired criteria - they generalize effectively between datasets and gi…
With the rapid development of generation model, AI-based face manipulation technology, which called DeepFakes, has become more and more realistic. This means of face forgery can attack any target, which poses a new threat to personal privacy and property security. Moreover, the misuse of synthetic video shows potential dangers in many areas, such as identity harassment, pornography and news rumors. Inspired by the fact that the spatial coherence and temporal consistency of physiological signal are destroyed in the generated content, we attempt to find inconsistent patterns that can distinguish…
Contrastive Desensitization Learning for Cross Domain Face Forgery Detection
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics & MIIT Key Laboratory of Pattern Analysis and Machine Intelligence（计算机科学与技术学院，南京航空航天大学及信息通信技术部模式分析与机器智能重点实验室）
TT-DF: A Large-Scale Diffusion-Based Dataset and Benchmark for Human Body Forgery Detection
MAIS \& NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China University of Science
COMICS: End-to-end Bi-grained Contrastive Learning for Multi-face Forgery Detection
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Pengcheng Laboratory（鹏城实验室）；College of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；University at Buffalo, SUNY（纽约州立大学布法罗分校）
SGF-CDNet: A Consistency-Discrepancy Graph Network over Semantic-Geometric Fused Nodes for Face Forgery Detection
School of Cyber Science and Technology, University of Science and Technology of China（中国科学技术大学网络空间安全学院）；Anhui Province Key Laboratory of Digital Security（安徽省数字安全重点实验室）
As the remarkable development of facial manipulation technologies is accompanied by severe security concerns, face forgery detection has become a recent research hotspot. Most existing detection methods train a binary classifier under global supervision to judge real or fake. However, advanced manipulations only perform small-scale tampering, posing challenges to comprehensively capture subtle and local forgery artifacts, especially in high compression settings and cross-dataset scenarios. To address such limitations, we propose a novel framework named Multi-modal Contrastive Classification by…
LAKAN: Landmark-assisted Adaptive Kolmogorov-Arnold Network for Face Forgery Detection
School of Cyber Science and Technology, University of Science and Technology of China（计算机科学与技术学院，中国科学技术大学）；Anhui Province Key Laboratory of Digital Security（安徽省数字安全重点实验室）
Rethinking Image Forgery Detection via Soft Contrastive Learning and Unsupervised Clustering
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Guangdong Key Laboratory of Intelligent Information Processing, College of Electronics and Information Engineering, Shenzhen University（广东智能信息处理重点实验室、电子与信息工程学院）
One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common corruptions such as compression. In this paper, we examine whether we can tackle this issue by harnessing videos of real talking faces, which contain rich information on natural facial appearance and behaviour and are readily available in large quantities online. Our method, termed RealForensics, consists of two stages. First, we exploit the natural correspondence between the visual and auditory modalities in real vi…
Technical University of Munich（慕尼黑工业大学）；University Federico II of Naples（那不勒斯费德里科二世大学）；University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）
The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the be…
A Dual-Branch CNN for Robust Detection of AI-Generated Facial Forgeries
Computer Science Department University of Southern Maine Portland, United States（计算机科学系 俄亥俄州立大学 帕斯托市 美国）；The Department of Computer Science University of Central Oklahoma Edmond, United States（计算机科学系 中央俄克拉荷马大学 埃德蒙德 美国）
École des Ponts Paristech（巴黎高科桥梁学院）；JFLI, CNRS, UMI 3527（法国国家科学研究中心JFLI联合实验室）；LIGM, UMR 8049（计算机科学、建模与应用实验室）；UPEM（巴黎东马恩-拉瓦雷大学）；National Institute of Informatics（国立情报学研究所）
This paper presents a method to automatically and efficiently detect face tampering in videos, and particularly focuses on two recent techniques used to generate hyper-realistic forged videos: Deepfake and Face2Face. Traditional image forensics techniques are usually not well suited to videos due to the compression that strongly degrades the data. Thus, this paper follows a deep learning approach and presents two networks, both with a low number of layers to focus on the mesoscopic properties of images. We evaluate those fast networks on both an existing dataset and a dataset we have constitut…
We propose Deep Distribution Transfer(DDT), a new transfer learning approach to address the problem of zero and few-shot transfer in the context of facial forgery detection. We examine how well a model (pre-)trained with one forgery creation method generalizes towards a previously unseen manipulation technique or different dataset. To facilitate this transfer, we introduce a new mixture model-based loss formulation that learns a multi-modal distribution, with modes corresponding to class categories of the underlying data of the source forgery method. Our core idea is to first pre-train an enco…
MFFI: Multi-Dimensional Face Forgery Image Dataset for Real-World Scenarios
Anhui Province Key Laboratory of Digital Security（安徽省级数字安全重点实验室）；Hefei University of Technology（合肥工业大学）；Cardiff University（卡迪夫大学）；IHPC, Agency for Science, Technology and Research（科技研究局信息处理中心）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
EvolveReason: Self-Evolving Reasoning Paradigm for Explainable Deepfake Facial Image Identification
School of Software Technology, Zhejiang University（浙江大学软件技术学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Ant Group（蚂蚁集团）
StegaFFD: Privacy-Preserving Face Forgery Detection via Fine-Grained Steganographic Domain Lifting
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；Beihang University（北京航空航天大学）；Macau University of Science and Technology（澳门科技大学）；Nanyang Technological University（南洋理工大学）
This paper presents an integration of image forgery detection with image facial recognition using black propagation neural network (BPNN). We observed that facial image recognition by itself will always give a matching output or closest possible output image for every input image irrespective of the authenticity or otherwise not of the testing input image. Based on this, we are proposing the combination of the blind but powerful automation image forgery detection for entire input images for the BPNN recognition program. Hence, an input image must first be authenticated before being fed into th…
Beijing University of Posts and Telecommunications（北京邮电大学）
Although vanilla Convolutional Neural Network (CNN) based detectors can achieve satisfactory performance on fake face detection, we observe that the detectors tend to seek forgeries on a limited region of face, which reveals that the detectors is short of understanding of forgery. Therefore, we propose an attention-based data augmentation framework to guide detector refine and enlarge its attention. Specifically, our method tracks and occludes the Top-N sensitive facial regions, encouraging the detector to mine deeper into the regions ignored before for more representative forgery. Especially,…
Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）
In this paper, we propose a Geometry-Contrastive Generative Adversarial Network (GC-GAN) for transferring continuous emotions across different subjects. Given an input face with certain emotion and a target facial expression from another subject, GC-GAN can generate an identity-preserving face with the target expression. Geometry information is introduced into cGANs as continuous conditions to guide the generation of facial expressions. In order to handle the misalignment across different subjects or emotions, contrastive learning is used to transform geometry manifold into an embedded semanti…
The University of Tokyo（东京大学）；National Institute of Informatics（国立信息学研究所）；Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）
Deep-learning-based technologies such as deepfakes ones have been attracting widespread attention in both society and academia, particularly ones used to synthesize forged face images. These automatic and professional-skill-free face manipulation technologies can be used to replace the face in an original image or video with any target object while maintaining the expression and demeanor. Since human faces are closely related to identity characteristics, maliciously disseminated identity manipulated videos could trigger a crisis of public trust in the media and could even have serious politica…
Imperial College London（帝国理工学院）；Shanghai Jiao Tong University（上海交通大学）
The success of most advanced facial expression recognition works relies heavily on large-scale annotated datasets. However, it poses great challenges in acquiring clean and consistent annotations for facial expression datasets. On the other hand, self-supervised contrastive learning has gained great popularity due to its simple yet effective instance discrimination training strategy, which can potentially circumvent the annotation issue. Nevertheless, there remain inherent disadvantages of instance-level discrimination, which are even more challenging when faced with complicated facial represe…
Context-aware TFL: A Universal Context-aware Contrastive Learning Framework for Temporal Forgery Localization
School of Computer Science and Engineering, MoE Key Laboratory of Information Technology, Guangdong Province Key Laboratory of Information Security Technology, Sun Yat-sen University（计算机科学与工程学院、信息科技关键实验室、广东信息安全技术重点实验室、中山大学）；State Key Laboratory of Mathematical Engineering and Advanced Computing（数学工程与先进计算国家重点实验室）；School of Cyber Science and Technology, Shenzhen Campus, Sun Yat-sen University（网络安全科学与技术学院、深圳校区、中山大学）
Current facial expression recognition systems demand an expensive re-training routine when deployed to different scenarios than they were trained for. Biasing them towards learning specific facial characteristics, instead of performing typical transfer learning methods, might help these systems to maintain high performance in different tasks, but with a reduced training effort. In this paper, we propose Contrastive Inhibitory Adaptati On (CIAO), a mechanism that adapts the last layer of facial encoders to depict specific affective characteristics on different datasets. CIAO presents an improve…
Learning Real Facial Concepts for Independent Deepfake Detection
School of Software, Shandong University（山东大学软件学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；Nanyang Technological University（南洋理工大学）
Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline
Xi’an Jiaotong University（西安交通大学）；Hefei University of Technology（合肥工业大学）；CSIRO（澳大利亚联邦科学与工业研究组织）；Northwestern Polytechnical University（西北工业大学）；University of Macau（澳门大学）
Contrastive Learning of Person-independent Representations for Facial Action Unit Detection
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；University of Chinese Academy of Sciences（中国科学院大学）；Peng Cheng Laboratory（鹏城实验室）
With the rapid development of facial manipulation techniques, face forgery detection has received considerable attention in digital media forensics due to security concerns. Most existing methods formulate face forgery detection as a classification problem and utilize binary labels or manipulated region masks as supervision. However, without considering the correlation between local regions, these global supervisions are insufficient to learn a generalized feature and prone to overfitting. To address this issue, we propose a novel perspective of face forgery detection via local relation learni…
SenseTime Research（商汤科技研究院）；University of Electronic Science and Technology of China（电子科技大学）；College of Software, Beihang University（北京航空航天大学软件学院）；Northwestern Polytechnical University（西北工业大学）
As realistic facial manipulation technologies have achieved remarkable progress, social concerns about potential malicious abuse of these technologies bring out an emerging research topic of face forgery detection. However, it is extremely challenging since recent advances are able to forge faces beyond the perception ability of human eyes, especially in compressed images and videos. We find that mining forgery patterns with the awareness of frequency could be a cure, as frequency provides a complementary viewpoint where either subtle forgery artifacts or compression errors could be well descr…
Hanoi University of Science and Technology（河内理工大学）；Griffith University（格里菲斯大学）；Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Deakin University（迪肯大学）；The University of Queensland（昆士兰大学）
In recent years, visual forgery has reached a level of sophistication that humans cannot identify fraud, which poses a significant threat to information security. A wide range of malicious applications have emerged, such as fake news, defamation or blackmailing of celebrities, impersonation of politicians in political warfare, and the spreading of rumours to attract views. As a result, a rich body of visual forensic techniques has been proposed in an attempt to stop this dangerous trend. In this paper, we present a benchmark that provides in-depth insights into visual forgery and visual forens…
University of Technology Sydney（悉尼科技大学）；Baidu Inc.（百度公司）；Zhejiang University（浙江大学）
This paper investigates unsupervised representation learning for facial expression analysis. We think Unsupervised Facial Expression Representation (UFER) deserves exploration and has the potential to address some key challenges in facial expression analysis, such as scaling, annotation bias, the discrepancy between discrete labels and continuous emotions, and model pre-training. Such motivated, we propose a UFER method with contrastive local warping (ContraWarping), which leverages the insight that the emotional expression is robust to current global transformation (affine transformation, col…
