Convolutional Neural Networks (CNNs) have proved very accurate in multiple computer vision image classification tasks that required visual inspection in the past (e.g., object recognition, face detection, etc.). Motivated by these astonishing results, researchers have also started using CNNs to cope with image forensic problems (e.g., camera model identification, tampering detection, etc.). However, in computer vision, image classification methods typically rely on visual cues easily detectable by human eyes. Conversely, forensic solutions rely on almost invisible traces that are often very subtle and lie in the fine details of the image under analysis. For this reason, training a CNN to solve a forensic task requires some special care, as common processing operations (e.g., resampling, compression, etc.) can strongly hinder forensic traces. In this work, we focus on the effect that JPEG has on CNN training considering different computer vision and forensic image classification problems. Specifically, we consider the issues that rise from JPEG compression and misalignment of the JPEG grid. We show that it is necessary to consider these effects when generating a training dataset in order to properly train a forensic detector not losing generalization capability, whereas it is almost possible to ignore these effects for computer vision tasks.
University of Warwick（华威大学）；Deakin University（迪肯大学）；Hangzhou Dianzi University（杭州电子科技大学）
Device fingerprints like sensor pattern noise (SPN) are widely used for provenance analysis and image authentication. Over the past few years, the rapid advancement in digital photography has greatly reshaped the pipeline of image capturing process on consumer-level mobile devices. The flexibility of camera parameter settings and the emergence of multi-frame photography algorithms, especially high dynamic range (HDR) imaging, bring new challenges to device fingerprinting. The subsequent study on these topics requires a new purposefully built image dataset. In this paper, we present the Warwick…
Xidian University（西安电子科技大学）；University of Sydney（悉尼大学）
As JPEG is the most widely used image format, the importance of tampering detection for JPEG images in blind forensics is self-evident. In this area, extracting effective statistical characteristics from a JPEG image for classification remains a challenge. Effective features are designed manually in traditional methods, suggesting that extensive labor-consuming research and derivation is required. In this paper, we propose a novel image tampering detection method based on deep multi-scale discriminative networks (MSD-Nets). The multi-scale module is designed to automatically extract multiple f…
Indian Institute of Technology Jammu（印度理工学院查谟分校）；Dhirubhai Ambani Institute of Information and Communication Technology（迪鲁巴伊·安巴尼信息与通信技术学院）；Indian Institute of Technology Kharagpur（印度理工学院克勒格布尔分校）
The unprecedented growth in the easy availability of photo-editing tools has endangered the power of digital images.An image was supposed to be worth more than a thousand words,but now this can be said only if it can be authenticated orthe integrity of the image can be proved to be intact. In thispaper, we propose a digital image forensic technique for JPEG images. It can detect any forgery in the image if the forged portion called a ghost image is having a compression quality different from that of the cover image. It is based on resaving the JPEG image at different JPEG qualities, and the de…
A large number of image forensics methods are available which are capable of identifying image tampering. But these techniques are not capable of addressing the anti-forensics method which is able to hide the trace of image tampering. In this paper anti-forensics method for digital image compression has been proposed. This anti-forensics method is capable of removing the traces of image compression. Additionally, technique is also able to remove the traces of blocking artifact that are left by image compression algorithms that divide an image into segments during compression process. This meth…
Detection of double JPEG compression is important to forensics analysis. A few methods were proposed based on convolutional neural networks (CNNs). These methods only accept inputs from pre-processed data, such as histogram features and/or decompressed images. In this paper, we present a CNN solution by using raw DCT (discrete cosine transformation) coefficients from JPEG images as input. Considering the DCT sub-band nature in JPEG, a multiple-branch CNN structure has been designed to reveal whether a JPEG format image has been doubly compressed. Comparing with previous methods, the proposed m…
Data hiding is one widely used approach for protecting authentication and ownership. Most multimedia content like images and videos are transmitted or saved in the compressed form. This kind of lossy compression, such as JPEG, can destroy the hidden data, which raises the need of robust data hiding. It is still an open challenge to achieve the goal of data hiding that can be against these compressions. Recently, deep learning has shown large success in data hiding, while non-differentiability of JPEG makes it challenging to train a deep pipeline for improving robustness against lossy compressi…
Fake news often involves semantic manipulations across modalities such as image, text, location etc and requires the development of multimodal semantic forensics for its detection. Recent research has centered the problem around images, calling it image repurposing -- where a digitally unmanipulated image is semantically misrepresented by means of its accompanying multimodal metadata such as captions, location, etc. The image and metadata together comprise a multimedia package. The problem setup requires algorithms to perform multimodal semantic forensics to authenticate a query multimedia pac…
Beijing Jiaotong University（北京交通大学）；Beijing Key Laboratory of Advanced Information Science and Network Technology（北京先进信息科学与网络技术重点实验室）；University of Siena（锡耶纳大学）
Transferability of adversarial examples is a key issue to apply this kind of attacks against multimedia forensics (MMF) techniques based on Deep Learning (DL) in a real-life setting. Adversarial example transferability, in fact, would open the way to the deployment of successful counter forensics attacks also in cases where the attacker does not have a full knowledge of the to-be-attacked system. Some preliminary works have shown that adversarial examples against CNN-based image forensics detectors are in general non-transferrable, at least when the basic versions of the attacks implemented in…
University of Catania（卡塔尼亚大学）；iCTLab Spinoff of University of Catania（卡塔尼亚大学iCTLab衍生公司）；Banca d’Italia（意大利银行）
Most recent style-transfer techniques based on generative architectures are able to obtain synthetic multimedia contents, or commonly called deepfakes, with almost no artifacts. Researchers already demonstrated that synthetic images contain patterns that can determine not only if it is a deepfake but also the generative architecture employed to create the image data itself. These traces can be exploited to study problems that have never been addressed in the context of deepfakes. To this aim, in this paper a first approach to investigate the image ballistics on deepfake images subject to style…
With the development of deep learning, convolutional neural networks (CNNs) have become widely used in multimedia forensics for tasks such as detecting and identifying image forgeries. Meanwhile, anti-forensic attacks have been developed to fool these CNN-based forensic algorithms. Previous anti-forensic attacks often were designed to remove forgery traces left by a single manipulation operation as opposed to a set of manipulations. Additionally, recent research has shown that existing anti-forensic attacks against forensic CNNs have poor transferability, i.e. they are unable to fool other for…
School of Information and Communication Engineering, University of Electronic Science and Technology of China（电子科技大学信息与通信工程学院）
High dynamic range (HDR) imaging has recently drawn much attention in multimedia community. In this paper, we proposed a HDR image forensics method based on convolutional neural network (CNN).To our best knowledge, this is the first time to apply deep learning method on HDR image forensics. The proposed algorithm uses CNN to distinguish HDR images generated by multiple low dynamic range (LDR) images from that expanded by single LDR image using inverse tone mapping (iTM). To do this, we learn the change of statistical characteristics extracted by the proposed CNN architectures and classify two …
In the last few years, generative adversarial networks (GAN) have shown tremendous potential for a number of applications in computer vision and related fields. With the current pace of progress, it is a sure bet they will soon be able to generate high-quality images and videos, virtually indistinguishable from real ones. Unfortunately, realistic GAN-generated images pose serious threats to security, to begin with a possible flood of fake multimedia, and multimedia forensic countermeasures are in urgent need. In this work, we show that each GAN leaves its specific fingerprint in the images it …
Plug to Place: Indoor Multimedia Geolocation from Electrical Sockets for Digital Investigation
Forensics and Security Research Group, School of Computer Science（犯罪学与安全研究组，计算机科学学院）；School of Engineering（工程学院）；Case Western Reserve University（凯斯西储大学）
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Guilin University of Electronic Technology（桂林电子科技大学）；University of Macau（澳门大学）；Sun Yat-sen University（中山大学）
Image forensics is a rising topic as the trustworthy multimedia content is critical for modern society. Like other vision-related applications, forensic analysis relies heavily on the proper image representation. Despite the importance, current theoretical understanding for such representation remains limited, with varying degrees of neglect for its key role. For this gap, we attempt to investigate the forensic-oriented image representation as a distinct problem, from the perspectives of theory, implementation, and application. Our work starts from the abstraction of basic principles that the …
ForensicZip: More Tokens are Better but Not Necessary in Forensic Vision-Language Models
Great Bay University（大湾大学）；Shenzhen University（深圳大学）；Harbin Institute of Technology（哈尔滨工业大学）；School of Cyber Science and Technology, Sun Yat-sen University（中山大学网络安全科学与技术学院）
University of Siena（锡耶纳大学）；Shenzhen University（深圳大学）
Estimating the primary quantization matrix of double JPEG compressed images is a problem of relevant importance in image forensics since it allows to infer important information about the past history of an image. In addition, the inconsistencies of the primary quantization matrices across different image regions can be used to localize splicing in double JPEG tampered images. Traditional model-based approaches work under specific assumptions on the relationship between the first and second compression qualities and on the alignment of the JPEG grid. Recently, a deep learning-based estimator c…
Deep Learning-based Compressed Domain Multimedia for Man and Machine: A Taxonomy and Application to Point Cloud Classification
Instituto Superior Técnico - Universidade de Lisboa（里斯本大学高等技术学院）；Instituto de Telecomunicações（电信研究所）；ESTG, Politécnico de Leiria（莱里亚理工学院ESTG）；Faculty of Computers and Information, South Valley University（南谷大学计算机与信息学院）
In this work, we present a deep learning-based approach for image tampering localization fusion. This approach is designed to combine the outcomes of multiple image forensics algorithms and provides a fused tampering localization map, which requires no expert knowledge and is easier to interpret by end users. Our fusion framework includes a set of five individual tampering localization methods for splicing localization on JPEG images. The proposed deep learning fusion model is an adapted architecture, initially proposed for the image restoration task, that performs multiple operations in paral…
The deep learning revolution incited by the 2012 Alexnet paper has been transformative for the field of computer vision. Many problems which were severely limited using classical solutions are now seeing unprecedented success. The rapid proliferation of deep learning methods has led to a sharp increase in their use in consumer and embedded applications. One consequence of consumer and embedded applications is lossy multimedia compression which is required to engineer the efficient storage and transmission of data in these real-world scenarios. As such, there has been increased interest in a de…
Forensic analyses of digital images rely heavily on the traces of in-camera and out-camera processes left on the acquired images. Such traces represent a sort of camera fingerprint. If one is able to recover them, by suppressing the high-level scene content and other disturbances, a number of forensic tasks can be easily accomplished. A notable example is the PRNU pattern, which can be regarded as a device fingerprint, and has received great attention in multimedia forensics. In this paper we propose a method to extract a camera model fingerprint, called noiseprint, where the scene content is …
Univ. of Notre Dame（圣母大学）；Univ. of Campinas（坎皮纳斯大学）
Departing from traditional digital forensics modeling, which seeks to analyze single objects in isolation, multimedia phylogeny analyzes the evolutionary processes that influence digital objects and collections over time. One of its integral pieces is provenance filtering, which consists of searching a potentially large pool of objects for the most related ones with respect to a given query, in terms of possible ancestors (donors or contributors) and descendants. In this paper, we propose a two-tiered provenance filtering approach to find all the potential images that might have contributed to…
PRNU-based image processing is a key asset in digital multimedia forensics. It allows for reliable device identification and effective detection and localization of image forgeries, in very general conditions. However, performance impairs significantly in challenging conditions involving low quality and quantity of data. These include working on compressed and cropped images, or estimating the camera PRNU pattern based on only a few images. To boost the performance of PRNU-based analyses in such conditions we propose to leverage the image noiseprint, a recently proposed camera-model fingerprin…
Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；University of California, Berkeley（加州大学伯克利分校）
This paper proposes a fundamental answer to a frequently asked question in multimedia computing and machine learning: Do artifacts from perceptual compression contribute to error in the machine learning process and if so, how much? Our approach to the problem is a reinterpretation of the Helmholtz Free Energy formula from physics to explain the relationship between content and noise when using sensors (such as cameras or microphones) to capture multimedia data. The reinterpretation allows a bit-measurement of the noise contained in images, audio, and video by combining a classifier with percep…
Technion - Israel Institute of Technology（以色列理工学院）
The ever-growing amounts of visual contents captured on a daily basis necessitate the use of lossy compression methods in order to save storage space and transmission bandwidth. While extensive research efforts are devoted to improving compression techniques, every method inevitably discards information. Especially at low bit rates, this information often corresponds to semantically meaningful visual cues, so that decompression involves significant ambiguity. In spite of this fact, existing decompression algorithms typically produce only a single output, and do not allow the viewer to explore …
Politecnico di Milano（米兰理工大学）；University Federico II of Naples（那不勒斯费德里科二世大学）；University of Notre Dame（圣母大学）；University of Campinas（坎皮纳斯大学）；Purdue University（普渡大学）
The widespread diffusion of synthetically generated content is a serious threat that needs urgent countermeasures. As a matter of fact, the generation of synthetic content is not restricted to multimedia data like videos, photographs or audio sequences, but covers a significantly vast area that can include biological images as well, such as western blot and microscopic images. In this paper, we focus on the detection of synthetically generated western blot images. These images are largely explored in the biomedical literature and it has been already shown they can be easily counterfeited with …
Manipal Institute of Technology, Manipal Academy of Higher Education（曼尼帕尔理工学院，曼尼帕尔高等教育学院）；Deakin University（迪肯大学）
Source camera identification tools assist image forensic investigators to associate an image in question with a suspect camera. Various techniques have been developed based on the analysis of the subtle traces left in the images during the acquisition. The Photo Response Non Uniformity (PRNU) noise pattern caused by sensor imperfections has been proven to be an effective way to identify the source camera. The existing literature suggests that the PRNU is the only fingerprint that is device-specific and capable of identifying the exact source device. However, the PRNU is susceptible to camera s…
Progressive Feedback-Enhanced Transformer for Image Forgery Localization
State Key Laboratory of Media Convergence and Communication（媒体融合与传播国家重点实验室）；Communication University of China（中国传媒大学）；School of Computer and Cyber Sciences（计算机与网络科学学院）
Synthetic Aperture Radar (SAR) images are a valuable asset for a wide variety of tasks. In the last few years, many websites have been offering them for free in the form of easy to manage products, favoring their widespread diffusion and research work in the SAR field. The drawback of these opportunities is that such images might be exposed to forgeries and manipulations by malicious users, raising new concerns about their integrity and trustworthiness. Up to now, the multimedia forensics literature has proposed various techniques to localize manipulations in natural photographs, but the integ…
School of Computer Science, Wuhan University（武汉大学计算机学院）
Detecting facial forgery images and videos is an increasingly important topic in multimedia forensics. As forgery images and videos are usually compressed into different formats such as JPEG and H264 when circulating on the Internet, existing forgery-detection methods trained on uncompressed data often suffer from significant performance degradation in identifying them. To solve this problem, we propose a novel anti-compression facial forgery detection framework, which learns a compression-insensitive embedding feature space utilizing both original and compressed forgeries. Specifically, our a…
Generalized Synthetic Image Detection with Enhanced RGB-Noise Representation Learning
School of Computer and Cyber Sciences, Communication University of China（中国传媒大学计算机与网络空间学院）；School of Information Engineering, Changsha Medical University（长沙医学院信息工程学院）；Department of Information Engineering, Beijing Institute of Graphic Communication（北京印刷学院信息工程学院）；Fujian Provincial Key Laboratory of Big Data Mining and Applications, Fujian University of Technology（福建工程学院福建省大数据挖掘与应用重点实验室）
A Lightweight and Effective Image Tampering Localization Network with Vision Mamba
School of Computer and Cyber Sciences, Communication University of China（中国传媒大学计算机与网络空间科学学院）；State Key Laboratory of Media Convergence and Communication, Communication University of China（中国传媒大学媒体融合与传播国家重点实验室）；School of Information Engineering, Changsha Medical University（长沙医学院信息工程学院）
