We propose PhaseForensics, a DeepFake (DF) video detection method that leverages a phase-based motion representation of facial temporal dynamics. Existing methods relying on temporal inconsistencies for DF detection present many advantages over the typical frame-based methods. However, they still show limited cross-dataset generalization and robustness to common distortions. These shortcomings are partially due to error-prone motion estimation and landmark tracking, or the susceptibility of the pixel intensity-based features to spatial distortions and the cross-dataset domain shifts. Our key insight to overcome these issues is to leverage the temporal phase variations in the band-pass components of the Complex Steerable Pyramid on face sub-regions. This not only enables a robust estimate of the temporal dynamics in these regions, but is also less prone to cross-dataset variations. Furthermore, the band-pass filters used to compute the local per-frame phase form an effective defense against the perturbations commonly seen in gradient-based adversarial attacks. Overall, with PhaseForensics, we show improved distortion and adversarial robustness, and state-of-the-art cross-dataset generalization, with 91.2% video-level AUC on the challenging CelebDFv2 (a recent state-of-the-art compares at 86.9%).
Norwegian University of Science and Technology（挪威科技大学）；University Federico II of Naples（那不勒斯费德里科二世大学）
A fast-paced development of DeepFake generation techniques challenge the detection schemes designed for known type DeepFakes. A reliable Deepfake detection approach must be agnostic to generation types, which can present diverse quality and appearance. Limited generalizability across different generation schemes will restrict the wide-scale deployment of detectors if they fail to handle unseen attacks in an open set scenario. We propose a new approach, Multi-Channel Xception Attention Pairwise Interaction (MCX-API), that exploits the power of pairwise learning and complementary information fro…
With advancements of deep learning techniques, it is now possible to generate super-realistic images and videos, i.e., deepfakes. These deepfakes could reach mass audience and result in adverse impacts on our society. Although lots of efforts have been devoted to detect deepfakes, their performance drops significantly on previously unseen but related manipulations and the detection generalization capability remains a problem. Motivated by the fine-grained nature and spatial locality characteristics of deepfakes, we propose Locality-Aware AutoEncoder (LAE) to bridge the generalization gap. In t…
Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection
The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳））；University at Buffalo, State University of New York（纽约州立大学布法罗分校）；Nanjing University of Information Science and Technology（南京信息工程大学）
Do You See What I Say? Generalizable Deepfake Detection based on Visual Speech Recognition
Machine Intelligence Group, Department of CS&IS, Birla Institute of Technology and Sciences, Pilani, India（机器智能组，计算机科学与信息系，比拉理工学院和科学学院，比拉，印度）；STARS team Inria Center at Université Côte d’Azur in Sophia Antipolis, France（法国索菲亚Antipolis大学的Inria中心的STARS团队）
Deepfake Forensics Adapter: A Dual-Stream Network for Generalizable Deepfake Detection
Shenzhen Technology University（深圳技术大学）；Singapore Institute of Technology（新加坡理工学院）；The University of Hong Kong（香港大学）；Nanyang Technological University（南洋理工大学）
DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳）计算机科学与技术学院）；S-Lab, Nanyang Technological University（南洋理工大学S-Lab）
Suppressing Gradient Conflict for Generalizable Deepfake Detection
School of Software, Shandong University（山东大学软件学院）；Quan Cheng Laboratory（全成实验室）；School of Computer Science, National University of Singapore（新加坡国立大学计算机科学学院）
Explicit Correlation Learning for Generalizable Cross-Modal Deepfake Detection
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, University of Chinese Academy of Sciences（中国科学院大学网络空间安全学院）；University at Buffalo, State University of New York（纽约州立大学布法罗分校）；Institute of Microelectronics, Chinese Academy of Sciences（中国科学院微电子研究所）
LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection
CVI SnT University of Luxembourg（卢森堡大学CVI SnT研究所）；Cristal Laboratory National School of Computer Sciences University of Manouba（马努巴大学国家计算机科学学院Cristal实验室）
Environment-Invariant Subspace Learning for Generalizable Deepfake Detection
Tianjin University of Technology（天津理工大学）；Shandong Artificial Intelligence Institute（山东人工智能研究院）；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））
FairForensics: Seeing Expressions and Parsing Demographics via Vision-Language Modeling for Generalizable Fair Deepfake Detection
Shenzhen University（深圳大学）；Shandong Artificial Intelligence Institute（山东省人工智能研究院）；Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东省科学院））；Tianjin University of Technology（天津理工大学）
How Generalizable are Deepfake Image Detectors? An Empirical Study
School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；College of Computer Science and Technology, Harbin Engineering University（哈尔滨工程大学计算机科学与技术学院）
RoGA: Towards Generalizable Deepfake Detection through Robust Gradient Alignment
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics（计算机科学与技术学院，南京航空航天大学）；MIIT Key Laboratory of Pattern Analysis and Machine Intelligence（信息科技部模式分析与机器智能重点实验室）
The University of Adelaide（阿德莱德大学）；Tencent AI Lab（腾讯人工智能实验室）
Recent studies in deepfake detection have yielded promising results when the training and testing face forgeries are from the same dataset. However, the problem remains challenging when one tries to generalize the detector to forgeries created by unseen methods in the training dataset. This work addresses the generalizable deepfake detection from a simple principle: a generalizable representation should be sensitive to diverse types of forgeries. Following this principle, we propose to enrich the "diversity" of forgeries by synthesizing augmented forgeries with a pool of forgery configurations…
Towards Generalizable Deepfake Detection via Forgery-aware Audio-Visual Adaptation: A Variational Bayesian Approach
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Department of New Networks, Pengcheng Laboratory（鹏城实验室网络部）；School of Cyber Science and Technology, Sun Yat-sen University（中山大学网络安全科学与技术学院）；School of Computer Science and Cyber Engineering, Guangzhou University（广州大学计算机科学与网络工程学院）；School of Cyberspace Science, Harbin Institute of Technology（哈尔滨工业大学网络空间科学学院）
Applications of deep learning to synthetic media generation allow the creation of convincing forgeries, called DeepFakes, with limited technical expertise. DeepFake detection is an increasingly active research area. In this paper, we analyze an existing DeepFake detection technique based on head pose estimation, which can be applied when fake images are generated with an autoencoder-based face swap. Existing literature suggests that this method is an effective DeepFake detector, and its motivating principles are attractively simple. With an eye towards using these principles to develop new Dee…
Data-Independent Operator: A Training-Free Artifact Representation Extractor for Generalizable Deepfake Detection
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；University of Nevada, Reno（内华达大学雷诺分校）；School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳）数据科学学院）
Towards Generalizable Deepfake Image Detection with Vision Transformers
Department of Electronics and Communication Engineering（电子与通信工程系）；Department of Information Technology（信息技术系）；Department of Electrical and Electronics Engineering（电气与电子工程系）；National Institute of Technology Karnataka (NITK), Surathkal - 575025, India（卡纳塔克国家理工学院（NITK），苏拉塔卡尔-575025，印度）
Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoning
School of Advanced Interdisciplinary Sciences (SAIS), University of Chinese Academy of Sciences（中国科学院大学先进交叉学科学院）；MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所MAIS）；Ant Group（蚂蚁集团）；Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Generalizable Deepfake Detection Based on Forgery-aware Layer Masking and Multi-artifact Subspace Decomposition
Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology（南京信息工程大学数字取证教育部工程研究中心）
A Multi-Domain Feature Fusion Framework for Generalizable Deepfake Detection Across Different Generators
School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST), Islamabad, Pakistan（巴基斯坦国立科技大学电气工程与计算机科学学院）；Department of Communication, Quality Management and Information Systems, Mid Sweden University, Ostersund Campus, Sweden（瑞典中部大学通信、质量管理和信息系统系，厄斯特松德校区）；Department of Computer Science, Electrical and Space Engineering, Lulea University of Technology, Luleå, Sweden（瑞典吕勒奥理工大学计算机科学、电气与空间工程系）
IIT Jodhpur（印度理工学院焦特布尔分校）；University at Buffalo（布法罗大学）；IIT Kharagpur（印度理工学院卡拉格普尔分校）
The rapid progress in the ease of creating and spreading ultra-realistic media over social platforms calls for an urgent need to develop a generalizable deepfake detection technique. It has been observed that current deepfake generation methods leave discriminative artifacts in the frequency spectrum of fake images and videos. Inspired by this observation, in this paper, we present a novel approach, termed as MD-CSDNetwork, for combining the features in the spatial and frequency domains to mine a shared discriminative representation for classifying \textit{deepfakes}. MD-CSDNetwork is a novel …
Towards Sustainable Universal Deepfake Detection with Frequency-Domain Masking
Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）；Faculty of Science and Technology (REALTEK), Norwegian University of Life Sciences (NMBU)（科学与技术学院（REALTEK）、挪威生命科学大学）
Unsupervised Multimodal Deepfake Detection Using Intra- and Cross-Modal Inconsistencies
USC Information Sciences Institute（南加州大学信息科学研究所）；USC Ming Hsieh Department of Electrical and Computer Engineering（南加州大学明氏电气与计算机工程学院）；Clemson University（克莱姆森大学）；Holcombe Department of Electrical and Computer Engineering（霍尔科姆电气与计算机工程学院）
SpectraNet: FFT-assisted Deep Learning Classifier for Deepfake Face Detection
Department of Electronic and Telecommunications Engineering, University of Moratuwa（电子与电信工程系，穆拉图瓦大学）；School of Electrical and Data Engineering, University of Technology Sydney（电气与数据工程学院，技术悉尼大学）
C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detection
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；School of Information Science and Engineering, Yanshan University（燕山大学信息科学与工程学院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳）数据科学学院）
