Although vanilla Convolutional Neural Network (CNN) based detectors can achieve satisfactory performance on fake face detection, we observe that the detectors tend to seek forgeries on a limited region of face, which reveals that the detectors is short of understanding of forgery. Therefore, we propose an attention-based data augmentation framework to guide detector refine and enlarge its attention. Specifically, our method tracks and occludes the Top-N sensitive facial regions, encouraging the detector to mine deeper into the regions ignored before for more representative forgery. Especially, our method is simple-to-use and can be easily integrated with various CNN models. Extensive experiments show that the detector trained with our method is capable to separately point out the representative forgery of fake faces generated by different manipulation techniques, and our method enables a vanilla CNN-based detector to achieve state-of-the-art performance without structure modification.
A Sanity Check for Multi-In-Domain Face Forgery Detection in the Real World
Peking University（北京大学）；Nanjing University（南京大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Wuhan University（武汉大学）；Stanford University（斯坦福大学）
Nowadays, forgery faces pose pressing security concerns over fake news, fraud, impersonation, etc. Despite the demonstrated success in intra-domain face forgery detection, existing detection methods lack generalization capability and tend to suffer from dramatic performance drops when deployed to unforeseen domains. To mitigate this issue, this paper designs a more general fake face detection model based on the vision transformer(ViT) architecture. In the training phase, the pretrained ViT weights are freezed, and only the Low-Rank Adaptation(LoRA) modules are updated. Additionally, the Single…
SenseTime Research（商汤科技研究院）；University of Electronic Science and Technology of China（电子科技大学）；College of Software, Beihang University（北京航空航天大学软件学院）；Northwestern Polytechnical University（西北工业大学）
As realistic facial manipulation technologies have achieved remarkable progress, social concerns about potential malicious abuse of these technologies bring out an emerging research topic of face forgery detection. However, it is extremely challenging since recent advances are able to forge faces beyond the perception ability of human eyes, especially in compressed images and videos. We find that mining forgery patterns with the awareness of frequency could be a cure, as frequency provides a complementary viewpoint where either subtle forgery artifacts or compression errors could be well descr…
With the rapid development of generation model, AI-based face manipulation technology, which called DeepFakes, has become more and more realistic. This means of face forgery can attack any target, which poses a new threat to personal privacy and property security. Moreover, the misuse of synthetic video shows potential dangers in many areas, such as identity harassment, pornography and news rumors. Inspired by the fact that the spatial coherence and temporal consistency of physiological signal are destroyed in the generated content, we attempt to find inconsistent patterns that can distinguish…
FakeRadar: Probing Forgery Outliers to Detect Unknown Deepfake Videos
Guilin University of Electronic Technology（桂林电子科技大学）；Pengcheng Laboratory（鹏城实验室）；Sun Yat-sen University（中山大学）；Guangxi Key Laboratory of Image and Graphic Intelligent Processing（广西图像与图形智能处理重点实验室）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东省大数据分析与处理重点实验室）
So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection
University of Liverpool, UK（利物浦大学）；Nanyang Technological University（南洋理工大学）；Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；The University of Sheffield（谢菲尔德大学）
SenseTime Research（商汤科技研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai AI Laboratory（上海人工智能实验室）；Beihang University（北京航空航天大学）；University of Science and Technology of China（中国科学技术大学）；Nanyang Technological University（南洋理工大学）
The rapid progress of photorealistic synthesis techniques has reached at a critical point where the boundary between real and manipulated images starts to blur. Thus, benchmarking and advancing digital forgery analysis have become a pressing issue. However, existing face forgery datasets either have limited diversity or only support coarse-grained analysis. To counter this emerging threat, we construct the ForgeryNet dataset, an extremely large face forgery dataset with unified annotations in image- and video-level data across four tasks: 1) Image Forgery Classification, including two-way (rea…
FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models
School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；Peking University Shenzhen Graduate School（北京大学深圳研究生院）；Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology（广东省超高清沉浸式媒体技术重点实验室）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）
Digital watermarking has been widely used to protect the copyright and integrity of multimedia data. Previous studies mainly focus on designing watermarking techniques that are robust to attacks of destroying the embedded watermarks. However, the emerging deep learning based image generation technology raises new open issues that whether it is possible to generate fake watermarked images for circumvention. In this paper, we make the first attempt to develop digital image watermark fakers by using generative adversarial learning. Suppose that a set of paired images of original and watermarked i…
University Federico II of Naples（那不勒斯费德里科二世大学）；Technical University of Munich（慕尼黑工业大学）；University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）
Distinguishing manipulated from real images is becoming increasingly difficult as new sophisticated image forgery approaches come out by the day. Naive classification approaches based on Convolutional Neural Networks (CNNs) show excellent performance in detecting image manipulations when they are trained on a specific forgery method. However, on examples from unseen manipulation approaches, their performance drops significantly. To address this limitation in transferability, we introduce Forensic-Transfer (FT). We devise a learning-based forensic detector which adapts well to new domains, i.e.…
DevFD: Developmental Face Forgery Detection by Learning Shared and Orthogonal LoRA Subspaces
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所MAIS部门）；China Mobile Financial Technology Co., Ltd.（中国移动金融科技有限公司）；CAIR, HKSIS, Chinese Academy of Sciences（中国科学院CAIR部门，香港创新技术研究所）；School of Computer Science and Engineering, the Faculty of Innovation Engineering, M.U.S.T（澳门科技大学计算机科学与工程学院，创新工程学院）
Various facial manipulation techniques have drawn serious public concerns in morality, security, and privacy. Although existing face forgery classifiers achieve promising performance on detecting fake images, these methods are vulnerable to adversarial examples with injected imperceptible perturbations on the pixels. Meanwhile, many face forgery detectors always utilize the frequency diversity between real and fake faces as a crucial clue. In this paper, instead of injecting adversarial perturbations into the spatial domain, we propose a frequency adversarial attack method against face forgery…
Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models
The University of Hong Kong（香港大学）；HKU Shanghai Intelligent Computing Research Center（港大上海智能计算研究中心）；Shanghai AI Laboratory（上海人工智能实验室）；Zhejiang Laboratory（浙江实验室）；Hangzhou Institute for Advanced Study（杭州高等研究院）；Zhejiang University（浙江大学）；Alibaba（阿里巴巴）；MEGVII Technology（旷视科技）
Stacking Brick by Brick: Aligned Feature Isolation for Incremental Face Forgery Detection
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School（北京大学深圳研究生院电子与计算机工程学院）；WeChat Vision, Tencent Inc.（腾讯公司微信视觉部）
SFNet: Fusion of Spatial and Frequency-Domain Features for Remote Sensing Image Forgery Detection
School of Geography and Remote Sensing, Guangzhou University（广州大学地理科学与遥感学院）；Xiamen University Of Technology（厦门理工学院）；University of Illinois at Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校）；CyberGIS Center for Advanced Digital and Spatial Studies, University of Illinois at Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校CyberGIS研究中心）；School of Geosciences and Info-Physics, Central South University（中南大学地球科学与信息物理学院）
CASIA（中国科学院自动化研究所）；Anhui University（安徽大学）；University of Chinese Academy of Sciences（中国科学院大学）；SenseTime Research（商汤科技研究院）
Recent studies have shown that the performance of forgery detection can be improved with diverse and challenging Deepfakes datasets. However, due to the lack of Deepfakes datasets with large variance in appearance, which can be hardly produced by recent identity swapping methods, the detection algorithm may fail in this situation. In this work, we provide a new identity swapping algorithm with large differences in appearance for face forgery detection. The appearance gaps mainly arise from the large discrepancies in illuminations and skin colors that widely exist in real-world scenarios. Howev…
AltFreezing for More General Video Face Forgery Detection
University of Science and Technology of China（中国科学技术大学）；Microsoft Research Asia（微软亚洲研究院）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）
University of Science and Technology of China（中国科学技术大学）
Existing face forgery detection methods usually treat face forgery detection as a binary classification problem and adopt deep convolution neural networks to learn discriminative features. The ideal discriminative features should be only related to the real/fake labels of facial images. However, we observe that the features learned by vanilla classification networks are correlated to unnecessary properties, such as forgery methods and facial identities. Such phenomenon would limit forgery detection performance especially for the generalization ability. Motivated by this, we propose a novel met…
FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge
School of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）数据科学学院）
Xidian University（西安电子科技大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
In recent years, with the rapid development of face editing and generation, more and more fake videos are circulating on social media, which has caused extreme public concerns. Existing face forgery detection methods based on frequency domain find that the GAN forged images have obvious grid-like visual artifacts in the frequency spectrum compared to the real images. But for synthesized videos, these methods only confine to single frame and pay little attention to the most discriminative part and temporal frequency clue among different frames. To take full advantage of the rich information in …
Adaptive thresholding pattern for fingerprint forgery detection
Laboratory of Wireless Communication and Signal Processing (WCSP), Faculty of Electrical Engineering, Sahand University of Technology, Iran.（无线通信与信号处理实验室（WCSP）、电气工程学院、沙赫德大学技术学院、伊朗）
With the rapid development of facial forgery techniques, forgery detection has attracted more and more attention due to security concerns. Existing approaches attempt to use frequency information to mine subtle artifacts under high-quality forged faces. However, the exploitation of frequency information is coarse-grained, and more importantly, their vanilla learning process struggles to extract fine-grained forgery traces. To address this issue, we propose a progressive enhancement learning framework to exploit both the RGB and fine-grained frequency clues. Specifically, we perform a fine-grai…
With the ubiquitous diffusion of social networks, images are becoming a dominant and powerful communication channel. Not surprisingly, they are also increasingly subject to manipulations aimed at distorting information and spreading fake news. In recent years, the scientific community has devoted major efforts to contrast this menace, and many image forgery detectors have been proposed. Currently, due to the success of deep learning in many multimedia processing tasks, there is high interest towards CNN-based detectors, and early results are already very promising. Recent studies in computer v…
With various facial manipulation techniques arising, face forgery detection has drawn growing attention due to security concerns. Previous works always formulate face forgery detection as a classification problem based on cross-entropy loss, which emphasizes category-level differences rather than the essential discrepancies between real and fake faces, limiting model generalization in unseen domains. To address this issue, we propose a novel face forgery detection framework, named Dual Contrastive Learning (DCL), which specially constructs positive and negative paired data and performs designe…
Semantic Contextualization of Face Forgery: A New Definition, Dataset, and Detection Method
City University of Hong Kong（香港城市大学）；Nanyang Technological University（南洋理工大学）；JD Explore Academy（京东探索研究院）；University at Buffalo, State University of New York（纽约州立大学布法罗分校）
One of the most pressing challenges for the detection of face-manipulated videos is generalising to forgery methods not seen during training while remaining effective under common corruptions such as compression. In this paper, we examine whether we can tackle this issue by harnessing videos of real talking faces, which contain rich information on natural facial appearance and behaviour and are readily available in large quantities online. Our method, termed RealForensics, consists of two stages. First, we exploit the natural correspondence between the visual and auditory modalities in real vi…
Low-rank Orthogonal Subspace Intervention for Generalizable Face Forgery Detection
Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology（数字取证工程研究中心，教育部，南京信息科技大学）
Peking University（北京大学）；Microsoft Research Asia（微软亚洲研究院）
In this paper we propose a novel image representation called face X-ray for detecting forgery in face images. The face X-ray of an input face image is a greyscale image that reveals whether the input image can be decomposed into the blending of two images from different sources. It does so by showing the blending boundary for a forged image and the absence of blending for a real image. We observe that most existing face manipulation methods share a common step: blending the altered face into an existing background image. For this reason, face X-ray provides an effective way for detecting forge…
Exploring the Adversarial Robustness of Face Forgery Detection with Decision-based Black-box Attacks
Shanghai Engineering Research Center of AI & Robotics, Academy for Engineering & Technology, Fudan University, Shanghai 200433, China（上海人工智能与机器人工程研究中心，工程与技术学院，复旦大学，上海）；Engineering Research Center of Robotics, Ministry of Education, Academy for Engineering & Technology, Fudan University, Shanghai 200433, China（机器人工程研究中心，教育部，工程与技术学院，复旦大学，上海）；Youtu Lab, Tencent, Shanghai 200000, China（优衣库实验室，腾讯，上海）；vivo Mobile Communication Co., Ltd, Shanghai 200120, China（vivo移动通信有限公司，上海）；Shanghai Key Lab of Intelligent Information Processing, School of Computer Science, Fudan University, Shanghai 200433, China（上海智能信息处理重点实验室，计算机科学学院，复旦大学，上海）
University of Science and Technology of China（中国科学技术大学）
Intra-frame inconsistency has been proved to be effective for the generalization of face forgery detection. However, learning to focus on these inconsistency requires extra pixel-level forged location annotations. Acquiring such annotations is non-trivial. Some existing methods generate large-scale synthesized data with location annotations, which is only composed of real images and cannot capture the properties of forgery regions. Others generate forgery location labels by subtracting paired real and fake images, yet such paired data is difficult to collected and the generated label is usuall…
University of Sydney（悉尼大学）；UNSW Canberra（新南威尔士大学堪培拉分校）；CSIRO Data61（澳大利亚联邦科学与工业研究组织Data61研究所）
Facial forgery detection is a crucial but extremely challenging topic, with the fast development of forgery techniques making the synthetic artefact highly indistinguishable. Prior works show that by mining both spatial and frequency information the forgery detection performance of deep learning models can be vastly improved. However, leveraging multiple types of information usually requires more than one branch in the neural network, which makes the model heavy and cumbersome. Knowledge distillation, as an important technique for efficient modelling, could be a possible remedy. We find that e…
In this paper, we propose a novel CycleGAN without checkerboard artifacts for counter-forensics of fake-image detection. Recent rapid advances in image manipulation tools and deep image synthesis techniques, such as Generative Adversarial Networks (GANs) have easily generated fake images, so detecting manipulated images has become an urgent issue. Most state-of-the-art forgery detection methods assume that images include checkerboard artifacts which are generated by using DNNs. Accordingly, we propose a novel CycleGAN without any checkerboard artifacts for counter-forensics of fake-mage detect…
Imperial College London（伦敦帝国学院）；Facebook London（伦敦脸书公司）
Although current deep learning-based face forgery detectors achieve impressive performance in constrained scenarios, they are vulnerable to samples created by unseen manipulation methods. Some recent works show improvements in generalisation but rely on cues that are easily corrupted by common post-processing operations such as compression. In this paper, we propose LipForensics, a detection approach capable of both generalising to novel manipulations and withstanding various distortions. LipForensics targets high-level semantic irregularities in mouth movements, which are common in many gener…
East China Normal University（华东师范大学）；Shanghai Industrial Control Safety Innovation Technology Co., LTD（上海工业控制安全创新科技有限公司）；Alibaba Group（阿里巴巴集团）；Nanyang Technological University（南洋理工大学）；Zhejiang Sci-Tech University（浙江理工大学）
Full face synthesis and partial face manipulation by virtue of the generative adversarial networks (GANs) and its variants have raised wide public concerns. In the multi-media forensics area, detecting and ultimately locating the image forgery has become an imperative task. In this work, we investigate the architecture of existing GAN-based face manipulation methods and observe that the imperfection of upsampling methods therewithin could be served as an important asset for GAN-synthesized fake image detection and forgery localization. Based on this basic observation, we have proposed a novel …
