We propose Deep Distribution Transfer(DDT), a new transfer learning approach to address the problem of zero and few-shot transfer in the context of facial forgery detection. We examine how well a model (pre-)trained with one forgery creation method generalizes towards a previously unseen manipulation technique or different dataset. To facilitate this transfer, we introduce a new mixture model-based loss formulation that learns a multi-modal distribution, with modes corresponding to class categories of the underlying data of the source forgery method. Our core idea is to first pre-train an encoder neural network, which maps each mode of this distribution to the respective class labels, i.e., real or fake images in the source domain by minimizing wasserstein distance between them. In order to transfer this model to a new domain, we associate a few target samples with one of the previously trained modes. In addition, we propose a spatial mixup augmentation strategy that further helps generalization across domains. We find this learning strategy to be surprisingly effective at domain transfer compared to a traditional classification or even state-of-the-art domain adaptation/few-shot learning methods. For instance, compared to the best baseline, our method improves the classification accuracy by 4.88% for zero-shot and by 8.38% for the few-shot case transferred from the FaceForensics++ to Dessa dataset.
University of Sydney（悉尼大学）；UNSW Canberra（新南威尔士大学堪培拉分校）；CSIRO Data61（澳大利亚联邦科学与工业研究组织Data61研究所）
Facial forgery detection is a crucial but extremely challenging topic, with the fast development of forgery techniques making the synthetic artefact highly indistinguishable. Prior works show that by mining both spatial and frequency information the forgery detection performance of deep learning models can be vastly improved. However, leveraging multiple types of information usually requires more than one branch in the neural network, which makes the model heavy and cumbersome. Knowledge distillation, as an important technique for efficient modelling, could be a possible remedy. We find that e…
TT-DF: A Large-Scale Diffusion-Based Dataset and Benchmark for Human Body Forgery Detection
MAIS \& NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China University of Science
LAKAN: Landmark-assisted Adaptive Kolmogorov-Arnold Network for Face Forgery Detection
School of Cyber Science and Technology, University of Science and Technology of China（计算机科学与技术学院，中国科学技术大学）；Anhui Province Key Laboratory of Digital Security（安徽省数字安全重点实验室）
Rapid progress in deep learning is continuously making it easier and cheaper to generate video forgeries. Hence, it becomes very important to have a reliable way of detecting these forgeries. This paper describes such an approach for various tampering scenarios. The problem is modelled as a per-frame binary classification task. We propose to use transfer learning from face recognition task to improve tampering detection on many different facial manipulation scenarios. Furthermore, in low resolution settings, where single frame detection performs poorly, we try to make use of neighboring frames…
Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline
Xi’an Jiaotong University（西安交通大学）；Hefei University of Technology（合肥工业大学）；CSIRO（澳大利亚联邦科学与工业研究组织）；Northwestern Polytechnical University（西北工业大学）；University of Macau（澳门大学）
MFFI: Multi-Dimensional Face Forgery Image Dataset for Real-World Scenarios
Anhui Province Key Laboratory of Digital Security（安徽省级数字安全重点实验室）；Hefei University of Technology（合肥工业大学）；Cardiff University（卡迪夫大学）；IHPC, Agency for Science, Technology and Research（科技研究局信息处理中心）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
Hanoi University of Science and Technology（河内理工大学）；Griffith University（格里菲斯大学）；Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Deakin University（迪肯大学）；The University of Queensland（昆士兰大学）
In recent years, visual forgery has reached a level of sophistication that humans cannot identify fraud, which poses a significant threat to information security. A wide range of malicious applications have emerged, such as fake news, defamation or blackmailing of celebrities, impersonation of politicians in political warfare, and the spreading of rumours to attract views. As a result, a rich body of visual forensic techniques has been proposed in an attempt to stop this dangerous trend. In this paper, we present a benchmark that provides in-depth insights into visual forgery and visual forens…
DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of California, Irvine（加州大学尔湾分校）；Washington University in St. Louis（圣路易斯华盛顿大学）；Duke University（杜克大学）；University of California, Davis（加州大学戴维斯分校）；New York University（纽约大学）；Georgetown University（乔治城大学）
DevFD: Developmental Face Forgery Detection by Learning Shared and Orthogonal LoRA Subspaces
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所MAIS部门）；China Mobile Financial Technology Co., Ltd.（中国移动金融科技有限公司）；CAIR, HKSIS, Chinese Academy of Sciences（中国科学院CAIR部门，香港创新技术研究所）；School of Computer Science and Engineering, the Faculty of Innovation Engineering, M.U.S.T（澳门科技大学计算机科学与工程学院，创新工程学院）
School of Computer Science, Wuhan University（武汉大学计算机学院）
Detecting facial forgery images and videos is an increasingly important topic in multimedia forensics. As forgery images and videos are usually compressed into different formats such as JPEG and H264 when circulating on the Internet, existing forgery-detection methods trained on uncompressed data often suffer from significant performance degradation in identifying them. To solve this problem, we propose a novel anti-compression facial forgery detection framework, which learns a compression-insensitive embedding feature space utilizing both original and compressed forgeries. Specifically, our a…
University of Science and Technology of China（中国科学技术大学）
Existing face forgery detection methods usually treat face forgery detection as a binary classification problem and adopt deep convolution neural networks to learn discriminative features. The ideal discriminative features should be only related to the real/fake labels of facial images. However, we observe that the features learned by vanilla classification networks are correlated to unnecessary properties, such as forgery methods and facial identities. Such phenomenon would limit forgery detection performance especially for the generalization ability. Motivated by this, we propose a novel met…
École des Ponts Paristech（巴黎高科桥梁学院）；JFLI, CNRS, UMI 3527（法国国家科学研究中心JFLI联合实验室）；LIGM, UMR 8049（计算机科学、建模与应用实验室）；UPEM（巴黎东马恩-拉瓦雷大学）；National Institute of Informatics（国立情报学研究所）
This paper presents a method to automatically and efficiently detect face tampering in videos, and particularly focuses on two recent techniques used to generate hyper-realistic forged videos: Deepfake and Face2Face. Traditional image forensics techniques are usually not well suited to videos due to the compression that strongly degrades the data. Thus, this paper follows a deep learning approach and presents two networks, both with a low number of layers to focus on the mesoscopic properties of images. We evaluate those fast networks on both an existing dataset and a dataset we have constitut…
The University of Sydney, Sydney, Australia（悉尼大学）；Xiamen University, Xiamen, China（厦门大学）；Hong Kong Polytechnic University, HongKong, China（香港理工大学）；National University of Singapore, Singapore, Singapore（新加坡国立大学）
FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learning
State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江区）区块链与数据安全研究院）
This paper presents an integration of image forgery detection with image facial recognition using black propagation neural network (BPNN). We observed that facial image recognition by itself will always give a matching output or closest possible output image for every input image irrespective of the authenticity or otherwise not of the testing input image. Based on this, we are proposing the combination of the blind but powerful automation image forgery detection for entire input images for the BPNN recognition program. Hence, an input image must first be authenticated before being fed into th…
University of Maryland College Park（马里兰大学帕克分校）；Johns Hopkins University（约翰斯·霍普金斯大学）
Manipulated videos, especially those where the identity of an individual has been modified using deep neural networks, are becoming an increasingly relevant threat in the modern day. In this paper, we seek to develop a generalizable, explainable solution to detecting these manipulated videos. To achieve this, we design a series of forgery detection systems that each focus on one individual part of the face. These parts-based detection systems, which can be combined and used together in a single architecture, meet all of our desired criteria - they generalize effectively between datasets and gi…
University of Sydney（悉尼大学）；UNSW Canberra（新南威尔士大学堪培拉分校）；CSIRO Data61（澳大利亚联邦科学与工业研究组织数据61研究所）
Forgery facial images and videos have increased the concern of digital security. It leads to the significant development of detecting forgery data recently. However, the data, especially the videos published on the Internet, are usually compressed with lossy compression algorithms such as H.264. The compressed data could significantly degrade the performance of recent detection algorithms. The existing anti-compression algorithms focus on enhancing the performance in detecting heavily compressed data but less consider the compression adaption to the data from various compression levels. We bel…
Samsung AI Cambridge（三星人工智能剑桥中心）；University of Nottingham（诺丁汉大学）；Queen Mary University London（伦敦玛丽女王大学）
What is the best way to learn a universal face representation? Recent work on Deep Learning in the area of face analysis has focused on supervised learning for specific tasks of interest (e.g. face recognition, facial landmark localization etc.) but has overlooked the overarching question of how to find a facial representation that can be readily adapted to several facial analysis tasks and datasets. To this end, we make the following 4 contributions: (a) we introduce, for the first time, a comprehensive evaluation benchmark for facial representation learning consisting of 5 important face ana…
Semantic face editing of real world facial images is an important application of generative models. Recently, multiple works have explored possible techniques to generate such modifications using the latent structure of pre-trained GAN models. However, such approaches often require training an encoder network and that is typically a time-consuming and resource intensive process. A possible alternative to such a GAN-based architecture can be styleALAE, a latent-space based autoencoder that can generate photo-realistic images of high quality. Unfortunately, the reconstructed image in styleALAE d…
With the rapid development of generation model, AI-based face manipulation technology, which called DeepFakes, has become more and more realistic. This means of face forgery can attack any target, which poses a new threat to personal privacy and property security. Moreover, the misuse of synthetic video shows potential dangers in many areas, such as identity harassment, pornography and news rumors. Inspired by the fact that the spatial coherence and temporal consistency of physiological signal are destroyed in the generated content, we attempt to find inconsistent patterns that can distinguish…
SGF-CDNet: A Consistency-Discrepancy Graph Network over Semantic-Geometric Fused Nodes for Face Forgery Detection
School of Cyber Science and Technology, University of Science and Technology of China（中国科学技术大学网络空间安全学院）；Anhui Province Key Laboratory of Digital Security（安徽省数字安全重点实验室）
StegaFFD: Privacy-Preserving Face Forgery Detection via Fine-Grained Steganographic Domain Lifting
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Future Technology, University of Chinese Academy of Sciences（中国科学院大学未来技术学院）；Beihang University（北京航空航天大学）；Macau University of Science and Technology（澳门科技大学）；Nanyang Technological University（南洋理工大学）
The University of Tokyo（东京大学）；National Institute of Informatics（国立信息学研究所）；Sun Yat-sen University（中山大学）；South China University of Technology（华南理工大学）
Deep-learning-based technologies such as deepfakes ones have been attracting widespread attention in both society and academia, particularly ones used to synthesize forged face images. These automatic and professional-skill-free face manipulation technologies can be used to replace the face in an original image or video with any target object while maintaining the expression and demeanor. Since human faces are closely related to identity characteristics, maliciously disseminated identity manipulated videos could trigger a crisis of public trust in the media and could even have serious politica…
One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common corruptions such as compression. In this paper, we examine whether we can tackle this issue by harnessing videos of real talking faces, which contain rich information on natural facial appearance and behaviour and are readily available in large quantities online. Our method, termed RealForensics, consists of two stages. First, we exploit the natural correspondence between the visual and auditory modalities in real vi…
Technical University of Munich（慕尼黑工业大学）；University Federico II of Naples（那不勒斯费德里科二世大学）；University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）
The rapid progress in synthetic image generation and manipulation has now come to a point where it raises significant concerns for the implications towards society. At best, this leads to a loss of trust in digital content, but could potentially cause further harm by spreading false information or fake news. This paper examines the realism of state-of-the-art image manipulations, and how difficult it is to detect them, either automatically or by humans. To standardize the evaluation of detection methods, we propose an automated benchmark for facial manipulation detection. In particular, the be…
A Dual-Branch CNN for Robust Detection of AI-Generated Facial Forgeries
Computer Science Department University of Southern Maine Portland, United States（计算机科学系 俄亥俄州立大学 帕斯托市 美国）；The Department of Computer Science University of Central Oklahoma Edmond, United States（计算机科学系 中央俄克拉荷马大学 埃德蒙德 美国）
EvolveReason: Self-Evolving Reasoning Paradigm for Explainable Deepfake Facial Image Identification
School of Software Technology, Zhejiang University（浙江大学软件技术学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Ant Group（蚂蚁集团）
Beijing University of Posts and Telecommunications（北京邮电大学）
Although vanilla Convolutional Neural Network (CNN) based detectors can achieve satisfactory performance on fake face detection, we observe that the detectors tend to seek forgeries on a limited region of face, which reveals that the detectors is short of understanding of forgery. Therefore, we propose an attention-based data augmentation framework to guide detector refine and enlarge its attention. Specifically, our method tracks and occludes the Top-N sensitive facial regions, encouraging the detector to mine deeper into the regions ignored before for more representative forgery. Especially,…
