Deep neural networks (DNNs) have been used in digital forensics to identify fake facial images. We investigated several DNN-based forgery forensics models (FFMs) to examine whether they are secure against adversarial attacks. We experimentally demonstrated the existence of individual adversarial perturbations (IAPs) and universal adversarial perturbations (UAPs) that can lead a well-performed FFM to misbehave. Based on iterative procedure, gradient information is used to generate two kinds of IAPs that can be used to fabricate classification and segmentation outputs. In contrast, UAPs are generated on the basis of over-firing. We designed a new objective function that encourages neurons to over-fire, which makes UAP generation feasible even without using training data. Experiments demonstrated the transferability of UAPs across unseen datasets and unseen FFMs. Moreover, we conducted subjective assessment for imperceptibility of the adversarial perturbations, revealing that the crafted UAPs are visually negligible. These findings provide a baseline for evaluating the adversarial security of FFMs.
Deepfake Forensics Adapter: A Dual-Stream Network for Generalizable Deepfake Detection
Shenzhen Technology University（深圳技术大学）；Singapore Institute of Technology（新加坡理工学院）；The University of Hong Kong（香港大学）；Nanyang Technological University（南洋理工大学）
Whether, Which, and Whose: Solving the Triple Challenge of Deepfake Proactive Forensics in Multi-Face Scenarios
College of Computer Science and Technology（计算机科学与技术学院）；Xinjiang University（新疆大学）；College of Computer Science and Electronic Engineering（计算机科学与电子工程学院）；Hunan University（湖南大学）
ByteDance AI Lab（字节跳动人工智能实验室）；Harbin Institute of Technology（哈尔滨工业大学）
With the progress in AI-based facial forgery (i.e., deepfake), people are increasingly concerned about its abuse. Albeit effort has been made for training classification (also known as deepfake detection) models to recognize such forgeries, existing models suffer from poor generalization to unseen forgery technologies and high sensitivity to changes in image/video quality. In this paper, we advocate adversarial training for improving the generalization ability to both unseen facial forgeries and unseen image/video qualities. We believe training with samples that are adversarially crafted to at…
University of Macau（澳门大学）；Macau University of Science and Technology（澳门科技大学）
With the continuous research on Deepfake forensics, recent studies have attempted to provide the fine-grained localization of forgeries, in addition to the coarse classification at the video-level. However, the detection and localization performance of existing Deepfake forensic methods still have plenty of room for further improvement. In this work, we propose a Spatial-Temporal Deepfake Detection and Localization (ST-DDL) network that simultaneously explores spatial and temporal features for detecting and localizing forged regions. Specifically, we design a new Anchor-Mesh Motion (AMM) algor…
Deepfake face not only violates the privacy of personal identity, but also confuses the public and causes huge social harm. The current deepfake detection only stays at the level of distinguishing true and false, and cannot trace the original genuine face corresponding to the fake face, that is, it does not have the ability to trace the source of evidence. The deepfake countermeasure technology for judicial forensics urgently calls for deepfake traceability. This paper pioneers an interesting question about face deepfake, active forensics that "know it and how it happened". Given that deepfake…
University of Chinese Academy of Sciences（中国科学院大学）；Center for Research on Intelligent Perception and Computing, CASIA（中国科学院自动化研究所智能感知与计算研究中心）
Images synthesized by powerful generative adversarial network (GAN) based methods have drawn moral and privacy concerns. Although image forensic models have reached great performance in detecting fake images from real ones, these models can be easily fooled with a simple adversarial attack. But, the noise adding adversarial samples are also arousing suspicion. In this paper, instead of adding adversarial noise, we optimally search adversarial points on face manifold to generate anti-forensic fake face images. We iteratively do a gradient-descent with each small step in the latent space of a ge…
GIFGuard: Proactive Forensics against Deepfakes in Facial GIFs via Spatiotemporal Watermarking
School of Computer Science and Technology, Xinjiang University（新疆大学计算机科学与技术学院）；College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）
Pennsylvania State University（宾夕法尼亚州立大学）；Zhejiang University（浙江大学）；Shandong University（山东大学）
Facial Liveness Verification (FLV) is widely used for identity authentication in many security-sensitive domains and offered as Platform-as-a-Service (PaaS) by leading cloud vendors. Yet, with the rapid advances in synthetic media techniques (e.g., deepfake), the security of FLV is facing unprecedented challenges, about which little is known thus far. To bridge this gap, in this paper, we conduct the first systematic study on the security of FLV in real-world settings. Specifically, we present LiveBugger, a new deepfake-powered attack framework that enables customizable, automated security eva…
Embedding Aggregation for Forensic Facial Comparison
University of Brasilia（巴西利亚大学）；National Institute of Criminalistics（国家刑事科学研究所）；University of Beira Interior（贝拉内瓦大学）；Netherlands Forensic Institute（荷兰法医研究所）
National Institute of Standards and Technology（美国国家标准与技术研究院）
Traditionally, researchers in automatic face recognition and biometric technologies have focused on developing accurate algorithms. With this technology being integrated into operational systems, engineers and scientists are being asked, do these systems meet societal norms? The origin of this line of inquiry is `trust' of artificial intelligence (AI) systems. In this paper, we concentrate on adapting explainable AI to face recognition and biometrics, and we present four principles of explainable AI to face recognition and biometrics. The principles are illustrated by $\it{four}$ case studies,…
Hyper-realistic face image generation and manipulation have givenrise to numerous unethical social issues, e.g., invasion of privacy,threat of security, and malicious political maneuvering, which re-sulted in the development of recent deepfake detection methods with the rising demands of deepfake forensics. Proposed deepfake detection methods to date have shown remarkable detection performance and robustness. However, none of the suggested deepfake detection methods assessed the performance of deepfakes with the facemask during the pandemic crisis after the outbreak of theCovid-19. In this pap…
RheinMain University of Applied Sciences（莱茵美因应用科学大学）；Bielefeld University（比勒费尔德大学）；University Medical Center of the Johannes Gutenberg University Mainz（美因茨约翰内斯·古腾堡大学医学中心）；Johannes Gutenberg University Mainz（美因茨约翰内斯·古腾堡大学）
In this paper, we present a method for automated estimation of a human face given a skull remain. The proposed method is based on three statistical models. A volumetric (tetrahedral) skull model encoding the variations of different skulls, a surface head model encoding the head variations, and a dense statistic of facial soft tissue thickness (FSTT). All data are automatically derived from computed tomography (CT) head scans and optical face scans. In order to obtain a proper dense FSTT statistic, we register a skull model to each skull extracted from a CT scan and determine the FSTT value for…
Computer Vision Center（计算机视觉中心）；Universitat Autònoma de Barcelona（巴塞罗那自治大学）；Mitek Systems, Inc.（Mitek系统公司）
Counterfeiting and piracy are a form of theft that has been steadily growing in recent years. Banknotes and identity documents are two common objects of counterfeiting. Aiming to detect these counterfeits, the present survey covers a wide range of anti-counterfeiting security features, categorizing them into three components: security substrate, security inks and security printing. respectively. From the computer vision perspective, we present works in the literature covering these three categories. Other topics, such as history of counterfeiting, effects on society and document experts, count…
University of Brasilia（巴西利亚大学）；Federal University of Maranhao（马拉尼昂联邦大学）；KU Leuven（鲁汶大学）；National Institute of Criminalistics（国家刑事科学研究所）；Brazilian Federal Police（巴西联邦警察）
Facial analysis permits many investigations some of the most important of which are craniofacial identification, facial recognition, and age and sex estimation. In forensics, photo-anthropometry describes the study of facial growth and allows the identification of patterns in facial skull development by using a group of cephalometric landmarks to estimate anthropological information. In several areas, automation of manual procedures has achieved advantages over and similar measurement confidence as a forensic expert. This manuscript presents an approach using photo-anthropometric indexes, gene…
Face sketches are able to capture the spatial topology of a face while lacking some facial attributes such as race, skin, or hair color. Existing sketch-photo recognition approaches have mostly ignored the importance of facial attributes. In this paper, we propose a new loss function, called attribute-centered loss, to train a Deep Coupled Convolutional Neural Network (DCCNN) for the facial attribute guided sketch to photo matching. Specifically, an attribute-centered loss is proposed which learns several distinct centers, in a shared embedding space, for photos and sketches with different com…
Hanoi University of Science and Technology（河内理工大学）；Griffith University（格里菲斯大学）；Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Deakin University（迪肯大学）；The University of Queensland（昆士兰大学）
In recent years, visual forgery has reached a level of sophistication that humans cannot identify fraud, which poses a significant threat to information security. A wide range of malicious applications have emerged, such as fake news, defamation or blackmailing of celebrities, impersonation of politicians in political warfare, and the spreading of rumours to attract views. As a result, a rich body of visual forensic techniques has been proposed in an attempt to stop this dangerous trend. In this paper, we present a benchmark that provides in-depth insights into visual forgery and visual forens…
University of British Columbia（英属哥伦比亚大学）；Department of Statistics, University of British Columbia（英属哥伦比亚大学统计系）；Department of Electrical and Computer Engineering, University of British Columbia（英属哥伦比亚大学电气与计算机工程系）；School of Information and Communications Engineering, Xi’an Jiaotong University（西安交通大学信息与通信工程学院）
Recently, generative adversarial networks (GANs) can generate photo-realistic fake facial images which are perceptually indistinguishable from real face photos, promoting research on fake face detection. Though fake face forensics can achieve high detection accuracy, their anti-forensic counterparts are less investigated. Here we explore more \textit{imperceptible} and \textit{transferable} anti-forensics for fake face imagery detection based on adversarial attacks. Since facial and background regions are often smooth, even small perturbation could cause noticeable perceptual impairment in fak…
School of Computer, Information and Mathematical Sciences（计算机、信息与数学科学学院）
In this thesis, a multimodal biometric, secure encrypted data and encrypted biometric encoded into the QR code-based biometric-passport authentication method is proposed for national security applications. Firstly, using the Extended Profile - Local Binary Patterns (EP-LBP), a Canny edge detector, and the Scale Invariant Feature Transform (SIFT) algorithm with Image File Information (IMFINFO) process, the facial mark size recognition is initially achieved. Secondly, by using the Active Shape Model (ASM) into Active Appearance Model (AAM) to follow the hand and infusion the hand geometry charac…
University of Brasilia（巴西利亚大学）；University of Campinas（坎皮纳斯大学）；University of São Paulo（圣保罗大学）；University of Granada（格拉纳达大学）；Department of Federal Police（联邦警察局）
Facial landmarks are employed in many research areas such as facial recognition, craniofacial identification, age and sex estimation among the most important. In the forensic field, the focus is on the analysis of a particular set of facial landmarks, defined as cephalometric landmarks. Previous works demonstrated that the descriptive adequacy of these anatomical references for an indirect application (photo-anthropometric description) increased the marking precision of these points, contributing to a greater reliability of these analyzes. However, most of them are performed manually and all o…
The Institute of Electronics, Information and Communication Engineers（电子信息通信工程师协会）
Recently, image forensics community has paied attention to the research on the design of effective algorithms based on deep learning technology and facts proved that combining the domain knowledge of image forensics and deep learning would achieve more robust and better performance than the traditional schemes. Instead of improving it, in this paper, the safety of deep learning based methods in the field of image forensics is taken into account. To the best of our knowledge, this is a first work focusing on this topic. Specifically, we experimentally find that the method using deep learning wo…
We present a new type of backdoor attack that exploits a vulnerability of convolutional neural networks (CNNs) that has been previously unstudied. In particular, we examine the application of facial recognition. Deep learning techniques are at the top of the game for facial recognition, which means they have now been implemented in many production-level systems. Alarmingly, unlike other commercial technologies such as operating systems and network devices, deep learning-based facial recognition algorithms are not presently designed with security requirements or audited for security vulnerabili…
Beijing Jiaotong University（北京交通大学）；Beijing Key Laboratory of Advanced Information Science and Network Technology（北京先进信息科学与网络技术重点实验室）；University of Siena（锡耶纳大学）
Transferability of adversarial examples is a key issue to apply this kind of attacks against multimedia forensics (MMF) techniques based on Deep Learning (DL) in a real-life setting. Adversarial example transferability, in fact, would open the way to the deployment of successful counter forensics attacks also in cases where the attacker does not have a full knowledge of the to-be-attacked system. Some preliminary works have shown that adversarial examples against CNN-based image forensics detectors are in general non-transferrable, at least when the basic versions of the attacks implemented in…
Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）；Idneo Inc.（Idneo公司）
This article explores a face recognition alternative which seeks to contribute to resolve current security vulnerabilities in most recognition architectures. Current low cost facial authentication software in the market can be fooled by a printed picture of a face due to the lack of depth information. The presented software creates a depth map of the face with the help of a stereo setup, offering a higher level of security than traditional recognition programs. Analysis of the person's identity and facial depth map are processed through deep convolutional neural networks, providing a secure lo…
With the development of deep learning technology, the facial manipulation system has become powerful and easy to use. Such systems can modify the attributes of the given facial images, such as hair color, gender, and age. Malicious applications of such systems pose a serious threat to individuals' privacy and reputation. Existing studies have proposed various approaches to protect images against facial manipulations. Passive defense methods aim to detect whether the face is real or fake, which works for posterior forensics but can not prevent malicious manipulation. Initiative defense methods …
FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learning
State Key Laboratory of Blockchain and Data Security（区块链与数据安全国家重点实验室）；Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security（杭州高新技术区（滨江区）区块链与数据安全研究院）
A Deep Learning Approach for Facial Attribute Manipulation and Reconstruction in Surveillance and Reconnaissance
Department of Networks and Digital Media（网络与数字媒体系）；Kingston University（金史密斯大学）；School of Computer Science and Engineering（计算机科学与工程学院）；University of Westminster（威斯敏斯特大学）
Multimedia University（多媒体大学）；University of Nottingham（诺丁汉大学）；Monash University Malaysia（莫纳什大学马来西亚分校）
Over the last few years, automatic facial micro-expression analysis has garnered increasing attention from experts across different disciplines because of its potential applications in various fields such as clinical diagnosis, forensic investigation and security systems. Advances in computer algorithms and video acquisition technology have rendered machine analysis of facial micro-expressions possible today, in contrast to decades ago when it was primarily the domain of psychiatrists where analysis was largely manual. Indeed, although the study of facial micro-expressions is a well-establishe…
A Novel Evolutionary Method for Automated Skull-Face Overlay in Computer-Aided Craniofacial Superimposition
Department of Computer Science and Artificial Intelligence, University of Granada（计算机科学与人工智能系，格拉纳达大学）；Andalusian Research Institute in Data Science and Computational Intelligence, University of Granada（数据科学与计算智能安达卢西亚研究机构，格拉纳达大学）；Panacea Cooperative Research S. Coop.（Panacea合作研究公司）；Department of Software Engineering, University of Granada（软件工程系，格拉纳达大学）
With the increasing widely spread digital media become using in most fields such as medical care, Oceanography, Exploration processing, security purpose, military fields and astronomy, evidence in criminals and more vital fields and then digital Images become have different appreciation values according to what is important of carried information by digital images. Due to the easy manipulation property of digital images (by proper computer software) makes us doubtful when are juries using digital images as forensic evidence in courts, especially, if the digital images are main evidence to demo…
University of Haifa（海法大学）；University of Kent（肯特大学）
One of the main challenges faced by Biometric-based authentication systems is the need to offer secure authentication while maintaining the privacy of the biometric data. Previous solutions, such as Secure Sketch and Fuzzy Extractors, rely on assumptions that cannot be guaranteed in practice, and often affect the authentication accuracy. In this paper, we introduce HoneyFaces: the concept of adding a large set of synthetic faces (indistinguishable from real) into the biometric "password file". This password inflation protects the privacy of users and increases the security of the system withou…
University of Science and Technology in China（中国科学技术大学）
Benefiting from the development of generative adversarial networks (GAN), facial manipulation has achieved significant progress in both academia and industry recently. It inspires an increasing number of entertainment applications but also incurs severe threats to individual privacy and even political security meanwhile. To mitigate such risks, many countermeasures have been proposed. However, the great majority methods are designed in a passive manner, which is to detect whether the facial images or videos are tampered after their wide propagation. These detection-based methods have a fatal l…
Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Center for Cyber Security, New York University Abu Dhabi（纽约大学阿布扎比分校网络安全中心）
Recent advances in Machine Learning (ML) have opened up new avenues for its extensive use in real-world applications. Facial recognition, specifically, is used from simple friend suggestions in social-media platforms to critical security applications for biometric validation in automated immigration at airports. Considering these scenarios, security vulnerabilities to such ML algorithms pose serious threats with severe outcomes. Recent work demonstrated that Deep Neural Networks (DNNs), typically used in facial recognition systems, are susceptible to backdoor attacks; in other words,the DNNs t…
Visually realistic GAN-generated facial images raise obvious concerns on potential misuse. Many effective forensic algorithms have been developed to detect such synthetic images in recent years. It is significant to assess the vulnerability of such forensic detectors against adversarial attacks. In this paper, we propose a new black-box attack method against GAN-generated image detectors. A novel contrastive learning strategy is adopted to train the encoder-decoder network based anti-forensic model under a contrastive loss function. GAN images and their simulated real counterparts are construc…
University of Brasília (UnB)（巴西利亚大学）；Federal University of São Carlos (UFSCar)（圣卡洛斯联邦大学）；University of Campinas (Unicamp)（坎皮纳斯大学）；São Paulo State University (Unesp)（圣保罗州立大学）
Ensuring the security of transactions is currently one of the major challenges that banking systems deal with. The usage of face for biometric authentication of users is attracting large investments from banks worldwide due to its convenience and acceptability by people, especially in cross-domain scenarios, in which facial images from ID documents are compared with digital self-portraits (selfies) for the automated opening of new checking accounts, e.g, or financial transactions authorization. Actually, the comparison of selfies and IDs has also been applied in another wide variety of tasks n…
