Masked Conditional Diffusion Model for Enhancing Deepfake Detection
Chengdu University of Information Technology（成都信息工程大学）；Purdue University in Indianapolis（印第安纳波利斯普渡大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；University at Albany, State University of New York (SUNY)（纽约州立大学奥尔巴尼分校）
Tiewen Chen, Shanmin Yang, Shu Hu, Zhenghan Fang, Ying Fu, Xi Wu, Xin Wang
Recent studies on deepfake detection have achieved promising results when training and testing faces are from the same dataset. However, their results severely degrade when confronted with forged samples that the model has not yet seen during training. In this paper, deepfake data to help detect deepfakes. this paper present we put a new insight into diffusion model-based data augmentation, and propose a Masked Conditional Diffusion Model (MCDM) for enhancing deepfake detection. It generates a variety of forged faces from a masked pristine one, encouraging the deepfake detection model to learn generic and robust representations without overfitting to special artifacts. Extensive experiments demonstrate that forgery images generated with our method are of high quality and helpful to improve the performance of deepfake detection models.
DFREC: DeepFake Identity Recovery Based on Identity-aware Masked Autoencoder
College of Cyber Security, Jinan University（暨南大学网络空间安全学院）；University of Macau（澳门大学）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电机与电子工程学院）
Wavelet-Driven Generalizable Framework for Deepfake Face Forgery Detection
International Institute of Information Technology Hyderabad（印度信息技术学院海得拉巴分校）；Indian Institute of Technology Hyderabad（印度理工学院海得拉巴分校）；Michigan State University（密歇根州立大学）
Self-control: A Better Conditional Mechanism for Masked Autoregressive Model
Nankai University（南开大学）；School of Statistics and Data Science（统计与数据科学学院）；Key Laboratory of Pure Mathematics and Combinatorics, Ministry of Education（教育部纯数学与组合数学重点实验室）
FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge
School of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）数据科学学院）
HOLA: Enhancing Audio-visual Deepfake Detection via Hierarchical Contextual Aggregations and Efficient Pre-training
Xi'an Jiaotong University（西安交通大学）；Zhengzhou University（郑州大学）；University of Science and Technology of China（中国科学技术大学）；Dalian University of Technology（大连理工大学）；Zhejiang Lab（浙江实验室）
Recently, productization of face recognition and identification algorithms have become the most controversial topic about ethical AI. As new policies around digital identities are formed, we introduce three face access models in a hypothetical social network, where the user has the power to only appear in photos they approve. Our approach eclipses current tagging systems and replaces unapproved faces with quantitatively dissimilar deepfakes. In addition, we propose new metrics specific for this task, where the deepfake is generated at random with a guaranteed dissimilarity. We explain access m…
DICE: Discrete Inversion Enabling Controllable Editing for Multinomial Diffusion and Masked Generative Models
Rutgers University（新泽西罗格斯大学）；MIT-IBM Watson AI Lab（MIT-IBM沃森人工智能实验室）；Red Hat AI Innovation（红帽AI创新）；Google DeepMind（谷歌DeepMind）；NYU（纽约大学）；Walmart Global Tech（沃尔玛全球技术）；NEC Labs America（NEC美国实验室）；Massachusetts Institute of Technology（麻省理工学院）；ANU（澳大利亚国立大学）；UT Arlington（德克萨斯大学阿灵顿分校）
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
The emergence of deepfake technologies has become a matter of social concern as they pose threats to individual privacy and public security. It is now of great significance to develop reliable deepfake detectors. However, with numerous face manipulation algorithms present, it is almost impossible to collect sufficient representative fake faces, and it is hard for existing detectors to generalize to all types of manipulation. Therefore, we turn to learn the distribution of real faces, and indirectly identify fake images that deviate from the real face distribution. In this study, we propose Rea…
MacDiff: Unified Skeleton Modeling with Masked Conditional Diffusion
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；School of Electronics Engineering and Computer Science, Peking University（北京大学电子工程与计算机科学学院）
Hyper-realistic face image generation and manipulation have givenrise to numerous unethical social issues, e.g., invasion of privacy,threat of security, and malicious political maneuvering, which re-sulted in the development of recent deepfake detection methods with the rising demands of deepfake forensics. Proposed deepfake detection methods to date have shown remarkable detection performance and robustness. However, none of the suggested deepfake detection methods assessed the performance of deepfakes with the facemask during the pandemic crisis after the outbreak of theCovid-19. In this pap…
CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decomposition
Chengdu University of Information Technology（成都信息工程大学）；University at Buffalo, SUNY（纽约州立大学布法罗分校）；Purdue University（普渡大学）；Microsoft Research Asia（微软亚洲研究院）；University at Albany, SUNY（纽约州立大学奥尔巴尼分校）
Zhejiang University（浙江大学）；EPFL（洛桑联邦理工学院）；King’s College London（伦敦国王学院）；Alibaba Group（阿里巴巴集团）
Differentiable architecture search (DARTS) has been a mainstream direction in automatic machine learning. Since the discovery that original DARTS will inevitably converge to poor architectures, recent works alleviate this by either designing rule-based architecture selection techniques or incorporating complex regularization techniques, abandoning the simplicity of the original DARTS that selects architectures based on the largest parametric value, namely $α$. Moreover, we find that all the previous attempts only rely on classification labels, hence learning only single modal information and l…
TALL: Thumbnail Layout for Deepfake Video Detection
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Cyber Security, UCAS（中国科学院大学网络空间安全学院）；CRIPAC & MAIS, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室与智能分析与计算研究所）；School of Artificial Intelligence, UCAS（中国科学院大学人工智能学院）；School of Communications and Information Engineering, NJUPT（南京邮电大学通信与信息工程学院）；OPPO Research Institute（OPPO研究院）
Masked Autoencoders (MAE) have been popular paradigms for large-scale vision representation pre-training. However, MAE solely reconstructs the low-level RGB signals after the decoder and lacks supervision upon high-level semantics for the encoder, thus suffering from sub-optimal learned representations and long pre-training epochs. To alleviate this, previous methods simply replace the pixel reconstruction targets of 75% masked tokens by encoded features from pre-trained image-image (DINO) or image-language (CLIP) contrastive learning. Different from those efforts, we propose to Mimic before R…
MonoMAE: Enhancing Monocular 3D Detection through Depth-Aware Masked Autoencoders
S-Lab, Nanyang Technological University（新加坡南洋理工大学S-Lab）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）；University of Chinese Academy of Sciences（中国科学院大学）
Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning
Paul C. Lauterbur Research Center for Biomedical Imaging（保罗·C·劳特伯生物医学成像研究中心）；Shenzhen Institute of Advanced Technology（深圳先进技术研究院）；Pengcheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Harvard University（哈佛大学）；Shanghai AI Lab（上海人工智能实验室）
A Spatial-Frequency Aware Multi-Scale Fusion Network for Real-Time Deepfake Detection
Qilu University of Technology (Shandong Academy of Sciences)（齐鲁工业大学（山东科学院））；Shandong Artificial Intelligence Institute, China（山东省人工智能研究院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；Key Laboratory of Computing Power Network（计算能力网络关键实验室）
DeepShield: Fortifying Deepfake Video Detection with Local and Global Forgery Analysis
Sun Yat-sen University（中山大学）；Pengcheng Laboratory（鹏城实验室）；Guilin University of Electronic Technology（桂林电子科技大学）；Guangdong Key Laboratory of Big Data Analysis and Processing（广东大数据分析与处理重点实验室）
Morphology-optimized Multi-Scale Fusion: Combining Local Artifacts and Mesoscopic Semantics for Deepfake Detection and Localization
The State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）
University of California San Diego（加州大学圣迭戈分校）；Mayo Clinic（妙佑医疗国际）
An optical microscopic examination of thinly cut stained tissue on glass slides prepared from a FFPE tissue blocks is the gold standard for tissue diagnostics. In addition, the diagnostic abilities and expertise of any pathologist is dependent on their direct experience with common as well as rarer variant morphologies. Recently, deep learning approaches have been used to successfully show a high level of accuracy for such tasks. However, obtaining expert-level annotated images is an expensive and time-consuming task and artificially synthesized histological images can prove greatly beneficial…
DeepFake face swapping presents a significant threat to online security and social media, which can replace the source face in an arbitrary photo/video with the target face of an entirely different person. In order to prevent this fraud, some researchers have begun to study the adversarial methods against DeepFake or face manipulation. However, existing works focus on the white-box setting or the black-box setting driven by abundant queries, which severely limits the practical application of these methods. To tackle this problem, we introduce a practical adversarial attack that does not requir…
C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detection
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；School of Information Science and Engineering, Yanshan University（燕山大学信息科学与工程学院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen)（香港中文大学（深圳）数据科学学院）
