Fake Generated Painting Detection via Frequency Analysis
Beihang University（北京航空航天大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Yong Bai, Yuanfang Guo, Jinjie Wei, Lin Lu, Rui Wang, Yunhong Wang
With the development of deep neural networks, digital fake paintings can be generated by various style transfer algorithms.To detect the fake generated paintings, we analyze the fake generated and real paintings in Fourier frequency domain and observe statistical differences and artifacts. Based on our observations, we propose Fake Generated Painting Detection via Frequency Analysis (FGPD-FA) by extracting three types of features in frequency domain. Besides, we also propose a digital fake painting detection database for assessing the proposed method. Experimental results demonstrate the excellence of the proposed method in different testing conditions.
Current GAN-based art generation methods produce unoriginal artwork due to their dependence on conditional input. Here, we propose Sketch-And-Paint GAN (SAPGAN), the first model which generates Chinese landscape paintings from end to end, without conditional input. SAPGAN is composed of two GANs: SketchGAN for generation of edge maps, and PaintGAN for subsequent edge-to-painting translation. Our model is trained on a new dataset of traditional Chinese landscape paintings never before used for generative research. A 242-person Visual Turing Test study reveals that SAPGAN paintings are mistaken …
Deep neural networks can generate images that are astonishingly realistic, so much so that it is often hard for humans to distinguish them from actual photos. These achievements have been largely made possible by Generative Adversarial Networks (GANs). While deep fake images have been thoroughly investigated in the image domain - a classical approach from the area of image forensics - an analysis in the frequency domain has been missing so far. In this paper, we address this shortcoming and our results reveal that in frequency space, GAN-generated images exhibit severe artifacts that can be ea…
We propose a universal image reconstruction method to represent detailed images purely from binary sparse edge and flat color domain. Inspired by the procedures of painting, our framework, based on generative adversarial network, consists of three phases: Imitation Phase aims at initializing networks, followed by Generating Phase to reconstruct preliminary images. Moreover, Refinement Phase is utilized to fine-tune preliminary images into final outputs with details. This framework allows our model generating abundant high frequency details from sparse input information. We also explore the def…
To detect GAN generated images, conventional supervised machine learning algorithms require collection of a number of real and fake images from the targeted GAN model. However, the specific model used by the attacker is often unavailable. To address this, we propose a GAN simulator, AutoGAN, which can simulate the artifacts produced by the common pipeline shared by several popular GAN models. Additionally, we identify a unique artifact caused by the up-sampling component included in the common GAN pipeline. We show theoretically such artifacts are manifested as replications of spectra in the f…
Technische Universität Braunschweig（布伦瑞克工业大学）；Technische Universität Berlin（柏林工业大学）
Generative adversarial networks (GANs) have made remarkable progress in synthesizing realistic-looking images that effectively outsmart even humans. Although several detection methods can recognize these deep fakes by checking for image artifacts from the generation process, multiple counterattacks have demonstrated their limitations. These attacks, however, still require certain conditions to hold, such as interacting with the detection method or adjusting the GAN directly. In this paper, we introduce a novel class of simple counterattacks that overcomes these limitations. In particular, we s…
SFNet: Fusion of Spatial and Frequency-Domain Features for Remote Sensing Image Forgery Detection
School of Geography and Remote Sensing, Guangzhou University（广州大学地理科学与遥感学院）；Xiamen University Of Technology（厦门理工学院）；University of Illinois at Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校）；CyberGIS Center for Advanced Digital and Spatial Studies, University of Illinois at Urbana–Champaign（伊利诺伊大学厄巴纳-香槟分校CyberGIS研究中心）；School of Geosciences and Info-Physics, Central South University（中南大学地球科学与信息物理学院）
The rapid progression of Generative Adversarial Networks (GANs) has raised a concern of their misuse for malicious purposes, especially in creating fake face images. Although many proposed methods succeed in detecting GAN-based synthetic images, they are still limited by the need for large quantities of the training fake image dataset and challenges for the detector's generalizability to unknown facial images. In this paper, we propose a new approach that explores the asynchronous frequency spectra of color channels, which is simple but effective for training both unsupervised and supervised l…
FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge
School of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Data Science, The Chinese University of Hong Kong, Shenzhen, China（香港中文大学（深圳）数据科学学院）
Universität Bonn（波恩大学）；Fraunhofer SCAI（弗劳恩霍夫算法与科学计算研究所）；University of Bonn（波恩大学）；Fraunhofer Center for Machine Learning（弗劳恩霍夫机器学习中心）；Fraunhofer ITWM（弗劳恩霍夫工业数学研究所）；Institute for Numerical Simulation（数值模拟研究所）
As neural networks become able to generate realistic artificial images, they have the potential to improve movies, music, video games and make the internet an even more creative and inspiring place. Yet, the latest technology potentially enables new digital ways to lie. In response, the need for a diverse and reliable method toolbox arises to identify artificial images and other content. Previous work primarily relies on pixel-space CNNs or the Fourier transform. To the best of our knowledge, synthesized fake image analysis and detection methods based on a multi-scale wavelet representation, l…
Chung-Ang University（中央大学）；University of Haifa（海法大学）
Generative adversarial network (GAN) is a framework for generating fake data using a set of real examples. However, GAN is unstable in the training stage. In order to stabilize GANs, the noise injection has been used to enlarge the overlap of the real and fake distributions at the cost of increasing variance. The diffusion (or smoothing) may reduce the intrinsic underlying dimensionality of data but it suppresses the capability of GANs to learn high-frequency information in the training procedure. Based on these observations, we propose a data representation for the GAN training, called noisy …
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；University of Mannheim（曼海姆大学）
Recent advances in deep generative models for photo-realistic images have led to high quality visual results. Such models learn to generate data from a given training distribution such that generated images can not be easily distinguished from real images by the human eye. Yet, recent work on the detection of such fake images pointed out that they are actually easily distinguishable by artifacts in their frequency spectra. In this paper, we propose to generate images according to the frequency distribution of the real data by employing a spectral discriminator. The proposed discriminator is li…
Xidian University（西安电子科技大学）；Chongqing University of Posts and Telecommunications（重庆邮电大学）
In recent years, with the rapid development of face editing and generation, more and more fake videos are circulating on social media, which has caused extreme public concerns. Existing face forgery detection methods based on frequency domain find that the GAN forged images have obvious grid-like visual artifacts in the frequency spectrum compared to the real images. But for synthesized videos, these methods only confine to single frame and pay little attention to the most discriminative part and temporal frequency clue among different frames. To take full advantage of the rich information in …
Rapid progress in adversarial learning has enabled the generation of realistic-looking fake visual content. To distinguish between fake and real visual content, several detection techniques have been proposed. The performance of most of these techniques however drops off significantly if the test and the training data are sampled from different distributions. This motivates efforts towards improving the generalization of fake detectors. Since current fake content generation techniques do not accurately model the frequency spectrum of the natural images, we observe that the frequency spectrum o…
VGMShield: Mitigating Misuse of Video Generative Models
University of Virginia（弗吉尼亚大学）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
Generative Adversarial Networks have got the researchers' attention due to their state-of-the-art performance in generating new images with only a dataset of the target distribution. It has been shown that there is a dissimilarity between the spectrum of authentic images and fake ones. Since the Fourier transform is a bijective mapping, saying that the model has a significant problem in learning the original distribution is a fair conclusion. In this work, we investigate the possible reasons for the mentioned drawback in the architecture and mathematical theory of the current GANs. Then we pro…
Universidad de Sevilla（塞维利亚大学）；Museo Nacional del Prado（普拉多国家博物馆）
In the forensic studies of painting masterpieces, the analysis of the support is of major importance. For plain weave fabrics, the densities of vertical and horizontal threads are used as main features, while angle deviations from the vertical and horizontal axis are also of help. These features can be studied locally through the canvas. In this work, deep learning is proposed as a tool to perform these local densities and angle studies. We trained the model with samples from 36 paintings by Velázquez, Rubens or Ribera, among others. The data preparation and augmentation are dealt with at a fi…
Generative adversarial networks built from deep convolutional neural networks (GANs) lack the ability to exactly replicate the high-frequency components of natural images. To alleviate this issue, we introduce two novel training techniques called frequency dropping (F-Drop) and frequency matching (F-Match). The key idea of F-Drop is to filter out unnecessary high-frequency components from the input images of the discriminators. This simple modification prevents the discriminators from being confused by perturbations of the high-frequency components. In addition, F-Drop makes the GANs focus on …
Boundary-aware Decoupled Flow Networks for Realistic Extreme Rescaling
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机与软件学院）；WeChat Pay Lab33, Tencent（腾讯微信支付实验室）；Research Center of Artificial Intelligence, Peng Cheng Laboratory（鹏城实验室人工智能研究中心）
SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skipping
University of Electronic Science and Technology of China（电子科技大学）；The Hong Kong University of Science and Technology（香港科技大学）；Central South University（中南大学）；National University of Singapore（新加坡国立大学）；Shanghai Jiaotong University（上海交通大学）
Vulnerabilities in AI-generated Image Detection: The Challenge of Adversarial Attacks
Hefei University of Technology（合肥工业大学）；University of Science and Technology of China（中国科学技术大学）；School of Computing and Information Technology（计算与信息学院）；Institute of Artificial Intelligence（人工智能研究院）；Great Bay University（大亚湾大学）
Frequency Bias Matters: Diving into Robust and Generalized Deep Image Forgery Detection
Faculty of Data Science, City University of Macau（数据科学学院，澳门城市大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）
Various facial manipulation techniques have drawn serious public concerns in morality, security, and privacy. Although existing face forgery classifiers achieve promising performance on detecting fake images, these methods are vulnerable to adversarial examples with injected imperceptible perturbations on the pixels. Meanwhile, many face forgery detectors always utilize the frequency diversity between real and fake faces as a crucial clue. In this paper, instead of injecting adversarial perturbations into the spatial domain, we propose a frequency adversarial attack method against face forgery…
University of Southern California（南加州大学）；California State University, San Bernardino（加州州立大学圣贝纳迪诺分校）
A novel method for detecting CNN-generated images, called Attentive PixelHop (or A-PixelHop), is proposed in this work. It has three advantages: 1) low computational complexity and a small model size, 2) high detection performance against a wide range of generative models, and 3) mathematical transparency. A-PixelHop is designed under the assumption that it is difficult to synthesize high-quality, high-frequency components in local regions. It contains four building modules: 1) selecting edge/texture blocks that contain significant high-frequency components, 2) applying multiple filter banks t…
Digital watermarking has been widely used to protect the copyright and integrity of multimedia data. Previous studies mainly focus on designing watermarking techniques that are robust to attacks of destroying the embedded watermarks. However, the emerging deep learning based image generation technology raises new open issues that whether it is possible to generate fake watermarked images for circumvention. In this paper, we make the first attempt to develop digital image watermark fakers by using generative adversarial learning. Suppose that a set of paired images of original and watermarked i…
Towards Sustainable Universal Deepfake Detection with Frequency-Domain Masking
Singapore University of Technology and Design (SUTD)（新加坡科技设计大学）；Faculty of Science and Technology (REALTEK), Norwegian University of Life Sciences (NMBU)（科学与技术学院（REALTEK）、挪威生命科学大学）
CAE-Net: Generalized Deepfake Image Detection using Convolution and Attention Mechanisms with Spatial and Frequency Domain Features
Department of Electrical and Electronic Engineering, BUET, Dhaka 1205, Bangladesh（电子与电气工程系，布里克塔大学（BUET），达卡1205，孟加拉国）；Department of Computer Science and Engineering BRAC University Dhaka 1212, Bangladesh（计算机科学与工程系，布拉克大学（BRAC University），达卡1212，孟加拉国）
IIT Jodhpur（印度理工学院焦特布尔分校）；University at Buffalo（布法罗大学）；IIT Kharagpur（印度理工学院卡拉格普尔分校）
The rapid progress in the ease of creating and spreading ultra-realistic media over social platforms calls for an urgent need to develop a generalizable deepfake detection technique. It has been observed that current deepfake generation methods leave discriminative artifacts in the frequency spectrum of fake images and videos. Inspired by this observation, in this paper, we present a novel approach, termed as MD-CSDNetwork, for combining the features in the spatial and frequency domains to mine a shared discriminative representation for classifying \textit{deepfakes}. MD-CSDNetwork is a novel …
Attackers may manipulate audio with the intent of presenting falsified reports, changing an opinion of a public figure, and winning influence and power. The prevalence of inauthentic multimedia continues to rise, so it is imperative to develop a set of tools that determines the legitimacy of media. We present a method that analyzes audio signals to determine whether they contain real human voices or fake human voices (i.e., voices generated by neural acoustic and waveform models). Instead of analyzing the audio signals directly, the proposed approach converts the audio signals into spectrogram…
When Eyes Betray AI: Social Gaze Consistency as a Semantic Cue for AI-Generated Image Detection
School of Computer Engineering（计算机工程学院）；Hoseo University（Hoseo大学）；School of Electronic Engineering（电子工程学院）；Soongsil University（Soongsil大学）；School of Computer Science（计算机科学学院）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Dodging DeepFake Detection via Implicit Spatial-Domain Notch Filtering
Nanyang Technological University（南洋理工大学）；New York University（纽约大学）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；Institute of High Performance Computing (IHPC)（高性能计算研究所）；Centre for Frontier AI Research (CFAR)（前沿人工智能研究中心）；East China Normal University（华东师范大学）；Shanghai Industrial Control Safety Innovation Technology Co., LTD（上海工业控制安全创新科技有限公司）
As the remarkable development of facial manipulation technologies is accompanied by severe security concerns, face forgery detection has become a recent research hotspot. Most existing detection methods train a binary classifier under global supervision to judge real or fake. However, advanced manipulations only perform small-scale tampering, posing challenges to comprehensively capture subtle and local forgery artifacts, especially in high compression settings and cross-dataset scenarios. To address such limitations, we propose a novel framework named Multi-modal Contrastive Classification by…
As ultra-realistic face forgery techniques emerge, deepfake detection has attracted increasing attention due to security concerns. Many detectors cannot achieve accurate results when detecting unseen manipulations despite excellent performance on known forgeries. In this paper, we are motivated by the observation that the discrepancies between real and fake videos are extremely subtle and localized, and inconsistencies or irregularities can exist in some critical facial regions across various information domains. To this end, we propose a novel pipeline, Cross-Domain Local Forensics (XDLF), fo…
Unveiling Deepfakes: A Frequency-Aware Triple Branch Network for Deepfake Detection
State Key Laboratory of Blockchain and Data Security, Zhejiang University（区块链与数据安全国家重点实验室，浙江大学）；College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；State Grid Blockchain Technology (Beijing) Co., Ltd.（国家电网区块链技术（北京）有限公司）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江）区块链与数据安全研究院）；Hangzhou City University（杭州城市学院）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）
