In the world of fake news and deepfakes, there have been an alarmingly large number of cases of images being tampered with and published in newspapers, used in court, and posted on social media for defamation purposes. Detecting these tampered images is an important task and one we try to tackle. In this paper, we focus on the methods to detect if an image has been tampered with using both Deep Learning and Image transformation methods and comparing the performances and robustness of each method. We then attempt to identify the tampered area of the image and predict the corresponding mask. Based on the results, suggestions and approaches are provided to achieve a more robust framework to detect and identify the forgeries.
Recently, many detection methods based on convolutional neural networks (CNNs) have been proposed for image splicing forgery detection. Most of these detection methods focus on the local patches or local objects. In fact, image splicing forgery detection is a global binary classification task that distinguishes the tampered and non-tampered regions by image fingerprints. However, some specific image contents are hardly retained by CNN-based detection networks, but if included, would improve the detection accuracy of the networks. To resolve these issues, we propose a novel network called dual-…
University of Shanghai for Science and Technology（上海理工大学）；Fudan University（复旦大学）
Visual noise is often regarded as a disturbance in image quality, whereas it can also provide a crucial clue for image-based forensic tasks. Conventionally, noise is assumed to comprise an additive Gaussian model to be estimated and then used to reveal anomalies. However, for real sensor noise, it should be modeled as signal-dependent noise (SDN). In this work, we apply SDN to splicing forgery localization tasks. Through statistical analysis of the SDN model, we assume that noise can be modeled as a Gaussian approximation for a certain brightness and propose a likelihood model for a noise leve…
Universiti Malaysia Terengganu（马来西亚登嘉楼大学）；Universiti Tecknologi Malaysia（马来西亚工艺大学）
Most of researches on image forensics have been mainly focused on detection of artifacts introduced by a single processing tool. They lead in the development of many specialized algorithms looking for one or more particular footprints under specific settings. Naturally, the performance of such algorithms are not perfect, and accordingly the provided output might be noisy, inaccurate and only partially correct. Furthermore, a forged image in practical scenarios is often the result of utilizing several tools available by image-processing software systems. Therefore, reliable tamper detection req…
University of Padova（帕多瓦大学）；Politecnico di Milano（米兰理工大学）
Forgery operations on video contents are nowadays within the reach of anyone, thanks to the availability of powerful and user-friendly editing software. Integrity verification and authentication of videos represent a major interest in both journalism (e.g., fake news debunking) and legal environments dealing with digital evidence (e.g., a court of law). While several strategies and different forensics traces have been proposed in recent years, latest solutions aim at increasing the accuracy by combining multiple detectors and features. This paper presents a video forgery localization framework…
Civil aviation college, Shenyang Aerospace University（沈阳航空航天大学民航学院）；Criminal Investigation Police University of China（中国刑事警察学院）；College of Information Science and Engineering, Northeastern University（东北大学信息科学与工程学院）；Institute of Automation Chinese Academy of Sciences（中国科学院自动化研究所）
Imaging is a sophisticated process combining a plenty of photovoltaic conversions, which lead to some spectral signatures beyond visual perception in the final images. Any manipulation against an original image will destroy these signatures and inevitably leave some traces in the final forgery. Therefore we present a novel optic-physical method to discriminate splicing edges from natural edges in a tampered image. First, we transform the forensic image from RGB into color space of S and o1o2. Then on the assumption of Dichromatic Reflection Model, edges in the image are discovered by composite…
Detection and Localization of Multiple Image Splicing Using MobileNet V1
Symbiosis Institute of Technology, Symbiosis International (Deemed University)（共生国际（ deemed）大学共生技术学院）；Symbiosis Centre for Applied Artificial Intelligence, Symbiosis International (Deemed University)（共生国际（ deemed）大学共生应用人工智能中心）
Mayachitra Inc（Mayachitra公司）；University of California, Riverside（加州大学河滨分校）；Naval Air Warfare Center Weapons Division（海军空战中心武器部）；Mayachitra Inc., Santa Barbara, California, USA（Mayachitra公司（加利福尼亚州圣巴巴拉））；University of California, Riverside, California, USA（加州大学河滨分校（加利福尼亚州））；Naval Air Warfare Center Weapons Division, China Lake, California, USA（海军空战中心武器部（加利福尼亚州中国湖））
Realistic image forgeries involve a combination of splicing, resampling, cloning, region removal and other methods. While resampling detection algorithms are effective in detecting splicing and resampling, copy-move detection algorithms excel in detecting cloning and region removal. In this paper, we combine these complementary approaches in a way that boosts the overall accuracy of image manipulation detection. We use the copy-move detection method as a pre-filtering step and pass those images that are classified as untampered to a deep learning based resampling detection framework. Experimen…
Image splicing is a common form of image forgery. Such alterations may leave no visual clues of tampering. In recent works camera characteristics consistency across the image has been used to establish the authenticity and integrity of digital images. Such constant camera characteristic properties are inherent from camera manufacturing processes and are unique. The majority of digital cameras are equipped with spherical lens and this introduces radial distortions on images. This aberration is often disturbed and fails to be consistent across the image, when an image is spliced. This paper desc…
Center for Computational & Data Sciences, Independent University, Bangladesh（孟加拉国独立大学计算与数据科学中心）；Sungkyunkwan University（成均馆大学）
Conventional forgery localizing methods usually rely on different forgery footprints such as JPEG artifacts, edge inconsistency, camera noise, etc., with cross-entropy loss to locate manipulated regions. However, these methods have the disadvantage of over-fitting and focusing on only a few specific forgery footprints. On the other hand, real-life manipulated images are generated via a wide variety of forgery operations and thus, leave behind a wide variety of forgery footprints. Therefore, we need a more general approach for image forgery localization that can work well on a variety of forger…
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；Beijing Key Laboratory of Advanced Information Science and Network Technology（北京市先进信息科学与网络技术重点实验室）；Department of Information Engineering and Mathematics, University of Siena（锡耶纳大学信息工程与数学系）
Detection of inconsistencies of double JPEG artefacts across different image regions is often used to detect local image manipulations, like image splicing, and to localize them. In this paper, we move one step further, proposing an end-to-end system that, in addition to detecting and localizing spliced regions, can also distinguish regions coming from different donor images. We assume that both the spliced regions and the background image have undergone a double JPEG compression, and use a local estimate of the primary quantization matrix to distinguish between spliced regions taken from diff…
Mayachitra, Inc.（Mayachitra公司）；UC Santa Barbara（加州大学圣塔芭芭拉分校）
Seam carving is a popular technique for content aware image retargeting. It can be used to deliberately manipulate images, for example, change the GPS locations of a building or insert/remove roads in a satellite image. This paper proposes a novel approach for detecting and localizing seams in such images. While there are methods to detect seam carving based manipulations, this is the first time that robust localization and detection of seam carving forgery is made possible. We also propose a seam localization score (SLS) metric to evaluate the effectiveness of localization. The proposed metho…
Image forgery detection is the task of detecting and localizing forged parts in tampered images. Previous works mostly focus on high resolution images using traces of resampling features, demosaicing features or sharpness of edges. However, a good detection method should also be applicable to low resolution images because compressed or resized images are common these days. To this end, we propose a Shallow Convolutional Neural Network(SCNN), capable of distinguishing the boundaries of forged regions from original edges in low resolution images. SCNN is designed to utilize the information of ch…
ActivityForensics: A Comprehensive Benchmark for Localizing Manipulated Activity in Videos
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；Jiangxi Provincial Key Laboratory of Multimedia Intelligent Processing（江西省多媒体智能处理重点实验室）；Faculty of Engineering, Shenzhen MSU-BIT University（深圳北理莫斯科大学工程系）；VinUniversity
Reinforced Multi-teacher Knowledge Distillation for Efficient General Image Forgery Detection and Localization
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Cyber Science and Technology, Sun Yat-sen University（中山大学网络空间科学与技术学院）；Peng Cheng Laboratory（鹏城实验室）；Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen University（深圳广东省智能信息处理重点实验室）
Cyberspace Research Inst. Shahid Beheshti University（沙希德贝赫什提大学网络空间研究所）
Since images are used as evidence in many cases, validation of digital images is essential. Copy-move forgery is a special kind of manipulation in which some parts of an image is copied and pasted into another part of the same image. Various methods have been proposed to detect copy-move forgery, which have achieved promising results. In previous methods, a binary mask determining the original and forged region is presented as the final result. However, it is not specified which part of the mask is the forged region. It should be noted that discriminating the original region from the duplicate…
National Institute of Informatics（国立信息学研究所）；The Graduate University for Advanced Studies (SOKENDAI)（综合研究大学院大学（SOKENDAI））；University of Tokyo（东京大学）
The proliferation of deepfake media is raising concerns among the public and relevant authorities. It has become essential to develop countermeasures against forged faces in social media. This paper presents a comprehensive study on two new countermeasure tasks: multi-face forgery detection and segmentation in-the-wild. Localizing forged faces among multiple human faces in unrestricted natural scenes is far more challenging than the traditional deepfake recognition task. To promote these new tasks, we have created the first large-scale dataset posing a high level of challenges that is designed…
Rethinking Image Forgery Detection via Soft Contrastive Learning and Unsupervised Clustering
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Guangdong Key Laboratory of Intelligent Information Processing, College of Electronics and Information Engineering, Shenzhen University（广东智能信息处理重点实验室、电子与信息工程学院）
The amount of digital imagery recorded has recently grown exponentially, and with the advancement of software, such as Photoshop or Gimp, it has become easier to manipulate images. However, most images on the internet have not been manipulated and any automated manipulation detection algorithm must carefully control the false alarm rate. In this paper we discuss a method to automatically detect local resampling using deep learning while controlling the false alarm rate using a-contrario analysis. The automated procedure consists of three primary steps. First, resampling features are calculated…
Pixel-Inconsistency Modeling for Image Manipulation Localization
Nanyang Technology University（南洋理工大学）；School of Electrical and Electronic Engineering（电子电气工程学院）；Rapid-Rich Object Search (ROSE) Lab（快速富对象搜索实验室）；School of Computer Science and Engineering（计算机科学与工程学院）；Sun Yat-sen University（中山大学）；City University of Hong Kong（香港城市大学）；University of Campinas（坎皮纳斯大学）；Institute of Computing（计算机研究所）；Artificial Intelligence Laboratory (Recod.ai)（人工智能实验室（Recod.ai））
UniShield: An Adaptive Multi-Agent Framework for Unified Forgery Image Detection and Localization
School of Electronic and Computer Engineering, Peking University（北京大学电子与计算机工程学院）；School of Future Technology, South China University of Technology（华南理工大学未来技术学院）；School of Electronic and Information Engineering, South China University of Technology（华南理工大学电子与信息工程学院）；Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University（北京大学深圳研究生院超高清沉浸媒体技术省重点实验室）
Progressive Decision-Making for Localizing Open-Ended AI-Generated Image Forgeries
School of Artificial Intelligence, University of Science and Technology Beijing（北京科技大学人工智能学院）；Institute of Artificial Intelligence, University of Science and Technology Beijing（北京科技大学人工智能研究院）；Key Laboratory of Intelligent Bionic Unmanned Systems, Ministry of Education, University of Science and Technology Beijing（北京科技大学智能仿生无人系统教育部重点实验室）
Indian Institute of Technology Guwahati（印度理工学院古瓦哈蒂分校）；Department of Electronics and Electrical Engineering（电子与电气工程系）；Department of Electronics and Electrical Engineering, Indian Institute of Technology Guwahati（电子与电气工程系，印度理工学院古瓦哈蒂分校）；Indian Institute of Technology Guwahati, Assam-781039, India（印度理工学院古瓦哈蒂分校，印度阿萨姆邦-781039）
This paper proposes a novel two-stream encoder-decoder network, which utilizes both the high-level and the low-level image features for precisely localizing forged regions in a manipulated image. This is motivated from the fact that the forgery creation process generally introduces both the high-level artefacts (e.g. unnatural contrast) and the low-level artefacts (e.g. noise inconsistency) to the forged images. In the proposed two-stream network, one stream learns the low-level manipulation-related features in the encoder side by extracting noise residuals through a set of high-pass filters i…
University of California, Riverside（加州大学河滨分校）；Mayachitra Inc.（Mayachitra公司）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
With advanced image journaling tools, one can easily alter the semantic meaning of an image by exploiting certain manipulation techniques such as copy-clone, object splicing, and removal, which mislead the viewers. In contrast, the identification of these manipulations becomes a very challenging task as manipulated regions are not visually apparent. This paper proposes a high-confidence manipulation localization architecture which utilizes resampling features, Long-Short Term Memory (LSTM) cells, and encoder-decoder network to segment out manipulated regions from non-manipulated ones. Resampli…
Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Traces
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机与通信工程学院）；Show Lab, National University of Singapore（新加坡国立大学Show实验室）；Rapid-Rich Object Search (ROSE) Lab, Nanyang Technological University（南洋理工大学快速丰富目标搜索实验室）；School of Cyber Science and Engineering, Wuhan University（武汉大学网络安全学院）
本文提出 Delocate 模型，通过随机遮挡 ROI 的人脸恢复与伪造掩码监督定位两阶段方法，同时提升未知域 Deepfake 视频检测和篡改区域定位性能。
