This chapter presents a framework for detecting fake regions by using various methods including watermarking technique and blind approaches. In particular, we describe current categories on blind approaches which can be divided into five: pixel-based techniques, format-based techniques, camera-based techniques, physically-based techniques and geometric-based techniques. Then we take a second look on the geometric-based techniques and further categorize them in detail. In the following section, the state-of-the-art methods involved in the geometric technique are elaborated.
Fake videos represent an important misinformation threat. While existing forensic networks have demonstrated strong performance on image forgeries, recent results reported on the Adobe VideoSham dataset show that these networks fail to identify fake content in videos. In this paper, we show that this is due to video coding, which introduces local variation into forensic traces. In response, we propose VideoFACT - a new network that is able to detect and localize a wide variety of video forgeries and manipulations. To overcome challenges that existing networks face when analyzing videos, our ne…
Detection and Localization of Image Forgeries using Resampling Features and Deep Learning
Mayachitra Inc.（Mayachitra公司）；University of California, Riverside（加州大学河滨分校）；Naval Air Warfare Center Weapons Division（海军航空作战中心武器部）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
Progressive Decision-Making for Localizing Open-Ended AI-Generated Image Forgeries
School of Artificial Intelligence, University of Science and Technology Beijing（北京科技大学人工智能学院）；Institute of Artificial Intelligence, University of Science and Technology Beijing（北京科技大学人工智能研究院）；Key Laboratory of Intelligent Bionic Unmanned Systems, Ministry of Education, University of Science and Technology Beijing（北京科技大学智能仿生无人系统教育部重点实验室）
National Chung Hsing University（中兴大学）；Yango University（阳光学院）
Recent advances in AI technology have made the forgery of digital images and videos easier, and it has become significantly more difficult to identify such forgeries. These forgeries, if disseminated with malicious intent, can negatively impact social and political stability, and pose significant ethical and legal challenges as well. Deepfake is a variant of auto-encoders that use deep learning techniques to identify and exchange images of a person's face in a picture or film. Deepfake can result in an erosion of public trust in digital images and videos, which has far-reaching effects on poli…
A Dual-Branch CNN for Robust Detection of AI-Generated Facial Forgeries
Computer Science Department University of Southern Maine Portland, United States（计算机科学系 俄亥俄州立大学 帕斯托市 美国）；The Department of Computer Science University of Central Oklahoma Edmond, United States（计算机科学系 中央俄克拉荷马大学 埃德蒙德 美国）
Image forensics is an increasingly relevant problem, as it can potentially address online disinformation campaigns and mitigate problematic aspects of social media. Of particular interest, given its recent successes, is the detection of imagery produced by Generative Adversarial Networks (GANs), e.g. `deepfakes'. Leveraging large training sets and extensive computing resources, recent work has shown that GANs can be trained to generate synthetic imagery which is (in some ways) indistinguishable from real imagery. We analyze the structure of the generating network of a popular GAN implementatio…
Detecting reliably copy-move forgeries is difficult because images do contain similar objects. The question is: how to discard natural image self-similarities while still detecting copy-moved parts as being "unnaturally similar"? Copy-move may have been performed after a rotation, a change of scale and followed by JPEG compression or the addition of noise. For this reason, we base our method on SIFT, which provides sparse keypoints with scale, rotation and illumination invariant descriptors. To discriminate natural descriptor matches from artificial ones, we introduce an a contrario method whi…
University of Tübingen（图宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）；Bosch Center for Artificial Intelligence（博世人工智能中心）
We address the task of open-world class-agnostic object detection, i.e., detecting every object in an image by learning from a limited number of base object classes. State-of-the-art RGB-based models suffer from overfitting the training classes and often fail at detecting novel-looking objects. This is because RGB-based models primarily rely on appearance similarity to detect novel objects and are also prone to overfitting short-cut cues such as textures and discriminative parts. To address these shortcomings of RGB-based object detectors, we propose incorporating geometric cues such as depth …
In the world of fake news and deepfakes, there have been an alarmingly large number of cases of images being tampered with and published in newspapers, used in court, and posted on social media for defamation purposes. Detecting these tampered images is an important task and one we try to tackle. In this paper, we focus on the methods to detect if an image has been tampered with using both Deep Learning and Image transformation methods and comparing the performances and robustness of each method. We then attempt to identify the tampered area of the image and predict the corresponding mask. Bas…
Rochester Institute of Technology（罗切斯特理工学院）；SRI International（斯坦福国际研究院）；Paige；Cornell Tech（康奈尔科技校区）
This paper presents an approach to detect out-of-context (OOC) objects in an image. Given an image with a set of objects, our goal is to determine if an object is inconsistent with the scene context and detect the OOC object with a bounding box. In this work, we consider commonly explored contextual relations such as co-occurrence relations, the relative size of an object with respect to other objects, and the position of the object in the scene. We posit that contextual cues are useful to determine object labels for in-context objects and inconsistent context cues are detrimental to determini…
City University of Hong Kong（香港城市大学）；Shenyang Institute of Automation, Chinese Academy of Sciences（中国科学院沈阳自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Hunan University（湖南大学）；Nanjing University of Information Science and Technology（南京信息工程大学）
With the proliferation of face image manipulation (FIM) techniques such as Face2Face and Deepfake, more fake face images are spreading over the internet, which brings serious challenges to public confidence. Face image forgery detection has made considerable progresses in exposing specific FIM, but it is still in scarcity of a robust fake face detector to expose face image forgeries under complex scenarios such as with further compression, blurring, scaling, etc. Due to the relatively fixed structure, convolutional neural network (CNN) tends to learn image content representations. However, CNN…
Detecting manipulated facial images and videos is an increasingly important topic in digital media forensics. As advanced face synthesis and manipulation methods are made available, new types of fake face representations are being created which have raised significant concerns for their use in social media. Hence, it is crucial to detect manipulated face images and localize manipulated regions. Instead of simply using multi-task learning to simultaneously detect manipulated images and predict the manipulated mask (regions), we propose to utilize an attention mechanism to process and improve th…
Detecting Unexpected Obstacles for Self-Driving Cars: Fusing Deep Learning and Geometric Modeling
Daimler AG R&D（戴姆勒股份公司研发部）；TU Dresden（德累斯顿工业大学）
检测小型道路危险，例如丢失的货物，是自动驾驶汽车的一项重要功能。我们解决了这个具有挑战性的，很少解决的问题与视觉系统，利用外观，上下文以及几何线索。为了利用外观和上下文线索，我们提出了一种新的基于深度学习的障碍物检测框架。在这里，使用完全卷积网络的变体来预测（i）自由空间，（ii）道路上意外障碍物和（iii）背景的逐像素语义标记。几何线索利用国家的最先进的检测方法，通过基于模型的统计假设检验预测立体输入图像的障碍。我们提出了一个原则性的贝叶斯框架融合的语义和立体的检测结果。中层Stixel表示用于以灵活、紧凑和鲁棒的方式描述障碍物。我们在Lost and Found数据集上评估了我们的新障碍物检测系统，其中包括高度仅为5 cm的障碍物的非常具有挑战性的场景。总体而言，我们报告了对最先进技术的重大改进，相对性能增益高达50%。特别是，我们实现了超过90%的检测率，距离达50米。我们的系统在我们的自动驾驶平台上以22 Hz运行。
DYMAPIA: A Multi-Domain Framework for Detecting AI-based Video Manipulation
School of Computing（计算机学院）；Georgia Southern University（乔治亚南方大学）；School of Computing Sciences & Computer Engineering（计算机科学与工程学院）；The University of Southern Mississippi（密苏里州立大学）
CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；University of Maryland, College Park（马里兰大学帕克分校）；University of Mannheim（曼海姆大学）
The rapid advances in deep generative models over the past years have led to highly {realistic media, known as deepfakes,} that are commonly indistinguishable from real to human eyes. These advances make assessing the authenticity of visual data increasingly difficult and pose a misinformation threat to the trustworthiness of visual content in general. Although recent work has shown strong detection accuracy of such deepfakes, the success largely relies on identifying frequency artifacts in the generated images, which will not yield a sustainable detection approach as generative models continu…
This thesis surveys the research in patch-based synthesis and algorithms for finding correspondences between small local regions of images. We additionally explore a large kind of applications of this new fast randomized matching technique. One of the algorithms we have studied in particular is PatchMatch, can find similar regions or "patches" of an image one to two orders of magnitude faster than previous techniques. The algorithmic program is driven by applying mathematical properties of nearest neighbors in natural images. It is observed that neighboring correspondences tend to be similar o…
Information Technology University（信息技术大学）；University of Central Florida（中佛罗里达大学）
As airborne vehicles are becoming more autonomous and ubiquitous, it has become vital to develop the capability to detect the objects in their surroundings. This paper attempts to address the problem of drones detection from other flying drones. The erratic movement of the source and target drones, small size, arbitrary shape, large intensity variations, and occlusion make this problem quite challenging. In this scenario, region-proposal based methods are not able to capture sufficient discriminative foreground-background information. Also, due to the extremely small size and complex motion of…
We present a learning-based method for detecting real and fake deepfake multimedia content. To maximize information for learning, we extract and analyze the similarity between the two audio and visual modalities from within the same video. Additionally, we extract and compare affective cues corresponding to perceived emotion from the two modalities within a video to infer whether the input video is "real" or "fake". We propose a deep learning network, inspired by the Siamese network architecture and the triplet loss. To validate our model, we report the AUC metric on two large-scale deepfake d…
Detecting and Explaining Fake News Short Videos with Multimodal Content and Real-World Evidence
AI and Social Good Lab, AI Media Centre（AI媒体中心人工智能与社会公益实验室）；Southern University of Science and Technology（南方科技大学）；Beijing Normal University（北京师范大学）；Hong Kong Baptist University（香港浸会大学）
