The Deepfake phenomenon has become very popular nowadays thanks to the possibility to create incredibly realistic images using deep learning tools, based mainly on ad-hoc Generative Adversarial Networks (GAN). In this work we focus on the analysis of Deepfakes of human faces with the objective of creating a new detection method able to detect a forensics trace hidden in images: a sort of fingerprint left in the image generation process. The proposed technique, by means of an Expectation Maximization (EM) algorithm, extracts a set of local features specifically addressed to model the underlying convolutional generative process. Ad-hoc validation has been employed through experimental tests with naive classifiers on five different architectures (GDWCT, STARGAN, ATTGAN, STYLEGAN, STYLEGAN2) against the CELEBA dataset as ground-truth for non-fakes. Results demonstrated the effectiveness of the technique in distinguishing the different architectures and the corresponding generation process.
Deepfake face not only violates the privacy of personal identity, but also confuses the public and causes huge social harm. The current deepfake detection only stays at the level of distinguishing true and false, and cannot trace the original genuine face corresponding to the fake face, that is, it does not have the ability to trace the source of evidence. The deepfake countermeasure technology for judicial forensics urgently calls for deepfake traceability. This paper pioneers an interesting question about face deepfake, active forensics that "know it and how it happened". Given that deepfake…
University of Wisconsin-Eau Claire（威斯康星大学欧克莱尔分校）；Minnesota State University, Mankato（明尼苏达州立大学曼卡托分校）
Social media is currently being used by many individuals online as a major source of information. However, not all information shared online is true, even photos and videos can be doctored. Deepfakes have recently risen with the rise of technological advancement and have allowed nefarious online users to replace one face with a computer generated face of anyone they would like, including important political and cultural figures. Deepfakes are now a tool to be able to spread mass misinformation. There is now an immense need to create models that are able to detect deepfakes and keep them from b…
University of Technology Sydney（悉尼科技大学）；Swinburne University of Technology（斯威本科技大学）；City University of Macau（澳门城市大学）
DeepFakes are raising significant social concerns. Although various DeepFake detectors have been developed as forensic countermeasures, these detectors are still vulnerable to attacks. Recently, a few attacks, principally adversarial attacks, have succeeded in cloaking DeepFake images to evade detection. However, these attacks have typical detector-specific designs, which require prior knowledge about the detector, leading to poor transferability. Moreover, these attacks only consider simple security scenarios. Less is known about how effective they are in high-level scenarios where either the…
Hunan University（湖南大学）；Nanjing University of Information Science and Technology（南京信息工程大学）
Residual-domain feature is very useful for Deepfake detection because it suppresses irrelevant content features and preserves key manipulation traces. However, inappropriate residual prediction will bring side effects on detection accuracy. In addition, residual-domain features are easily affected by image operations such as compression. Most existing works exploit either spatial-domain features or residual-domain features, while neglecting that two types of features are mutually correlated. In this paper, we propose a guided residuals network, namely GRnet, which fuses spatial-domain and resi…
Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Traces
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机与通信工程学院）；Show Lab, National University of Singapore（新加坡国立大学Show实验室）；Rapid-Rich Object Search (ROSE) Lab, Nanyang Technological University（南洋理工大学快速丰富目标搜索实验室）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络安全学院）
本文提出 Delocate 模型，通过随机遮挡 ROI 的人脸恢复与伪造掩码监督定位两阶段方法，同时提升未知域 Deepfake 视频检测和篡改区域定位性能。
Forgery Guided Learning Strategy with Dual Perception Network for Deepfake Cross-domain Detection
School of Computer Science and Technology, Xinjiang University, Urumqi, China（新疆大学计算机科学与技术学院）；Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center, Urumqi, China（新疆多模态智能处理与信息安全工程技术研究中心）；College of Computer Science and Electronic Engineering, Hunan University, Changsha, China（湖南大学计算机科学与电子工程学院）；Department of Computer Science, State University of New York, New Paltz, New York, USA（纽约州立大学新帕尔兹分校计算机科学系）
Recap: Detecting Deepfake Video with Unpredictable Tampered Traces via Recovering Faces and Mapping Recovered Faces
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络空间安全学院）
本文提出 Recap 深度伪造检测模型，通过随机掩蔽 ROI 恢复人脸并以恢复结果监督面部映射，放大真假视频差异，在多个标准基准场景中验证有效。
FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge
School of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）数据科学学院）
CAD: A General Multimodal Framework for Video Deepfake Detection via Cross-Modal Alignment and Distillation
Peking University（北京大学）；Shenzhen Graduate School（深圳研究生院）；University of Science and Technology of China（中国科学技术大学）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
This work introduces a novel DeepFake detection framework based on physiological measurement. In particular, we consider information related to the heart rate using remote photoplethysmography (rPPG). rPPG methods analyze video sequences looking for subtle color changes in the human skin, revealing the presence of human blood under the tissues. In this work we investigate to what extent rPPG is useful for the detection of DeepFake videos. The proposed fake detector named DeepFakesON-Phys uses a Convolutional Attention Network (CAN), which extracts spatial and temporal information from video fr…
Deepfakes are a recent off-the-shelf manipulation technique that allows anyone to swap two identities in a single video. In addition to Deepfakes, a variety of GAN-based face swapping methods have also been published with accompanying code. To counter this emerging threat, we have constructed an extremely large face swap video dataset to enable the training of detection models, and organized the accompanying DeepFake Detection Challenge (DFDC) Kaggle competition. Importantly, all recorded subjects agreed to participate in and have their likenesses modified during the construction of the face-s…
Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges
University of Nevada, Reno（内华达大学雷诺分校）；Centre for Frontier AI Research (CFAR)（前沿人工智能研究中心）；Institute of High Performance Computing (IHPC)（高性能计算研究所）；Centre for Advanced Technologies in Online Safety (CATOS)（网络安全先进技术中心）
ED$^4$: Explicit Data-level Debiasing for Deepfake Detection
School of computer science, Wuhan University（武汉大学计算机学院）；WeChat, Tencent Inc.（腾讯公司）；School of electronic and computer engineering, Peking University Shenzhen Graduate School（北京大学深圳研究生院电子与计算机工程学院）
Evidence Packing for Cross-Domain Image Deepfake Detection with LVLMs
Anhui University（安徽大学）；Hefei University of Technology（合肥工业大学）；IAI, Hefei Comprehensive National Science Center（合肥综合国家科学中心人工智能研究院）；United Arab Emirates University（阿拉伯联合酋长国大学）
Istituto Eulero（欧拉研究所）；Università della Svizzera italiana（意大利语区瑞士大学）
Generative Adversarial Networks (GANs) have paved the path towards entirely new media generation capabilities at the forefront of image, video, and audio synthesis. However, they can also be misused and abused to fabricate elaborate lies, capable of stirring up the public debate. The threat posed by GANs has sparked the need to discern between genuine content and fabricated one. Previous studies have tackled this task by using classical machine learning techniques, such as k-nearest neighbours and eigenfaces, which unfortunately did not prove very effective. Subsequent methods have focused on …
Towards Generalizable Deepfake Detection via Forgery-aware Audio-Visual Adaptation: A Variational Bayesian Approach
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Department of New Networks, Pengcheng Laboratory（鹏城实验室网络部）；School of Cyber Science and Technology, Sun Yat-sen University（中山大学网络安全科学与技术学院）；School of Computer Science and Cyber Engineering, Guangzhou University（广州大学计算机科学与网络工程学院）；School of Cyberspace Science, Harbin Institute of Technology（哈尔滨工业大学网络空间科学学院）
Suppressing Gradient Conflict for Generalizable Deepfake Detection
School of Software, Shandong University（山东大学软件学院）；Quan Cheng Laboratory（全成实验室）；School of Computer Science, National University of Singapore（新加坡国立大学计算机科学学院）
FakeTracer: Catching Face-swap DeepFakes via Implanting Traces in Training
University of Chinese Academy of Sciences（中国科学院大学）；College of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；University at Buffalo, SUNY（纽约州立大学布法罗学院）
Ensemble Deep Learning Approaches for AI-Altered Video Detection
Department of Mathematical and Computational Sciences, University of Toronto Mississauga（多伦多大学密西沙加分校数学与计算科学系）；Department of Mathematical and Computational Sciences, University of Toronto（多伦多大学数学与计算科学系）；University of Toronto（多伦多大学）
FakeChain: Exposing Shallow Cues in Multi-Step Deepfake Detection
Sungkyunkwan University Department of Artificial Intelligence Suwon Gyeonggi-do South Korea；Sungkyunkwan University Computer Science \& Engineering Department Suwon Gyeonggi-do South Korea；Sungkyunkwan University
Advances in Artificial Intelligence and Image Processing are changing the way people interacts with digital images and video. Widespread mobile apps like FACEAPP make use of the most advanced Generative Adversarial Networks (GAN) to produce extreme transformations on human face photos such gender swap, aging, etc. The results are utterly realistic and extremely easy to be exploited even for non-experienced users. This kind of media object took the name of Deepfake and raised a new challenge in the multimedia forensics field: the Deepfake detection challenge. Indeed, discriminating a Deepfake f…
University of Maryland College Park（马里兰大学帕克分校）；Johns Hopkins University（约翰斯·霍普金斯大学）
Manipulated videos, especially those where the identity of an individual has been modified using deep neural networks, are becoming an increasingly relevant threat in the modern day. In this paper, we seek to develop a generalizable, explainable solution to detecting these manipulated videos. To achieve this, we design a series of forgery detection systems that each focus on one individual part of the face. These parts-based detection systems, which can be combined and used together in a single architecture, meet all of our desired criteria - they generalize effectively between datasets and gi…
