3D High-Fidelity Mask Face Presentation Attack Detection Challenge
MUST（澳门科技大学）；Mininglamp Academy of Sciences, Mininglamp Technology（明略科技研究院，明略科技）；University of Oulu（奥卢大学）；NLPR, CASIA（中国科学院自动化研究所模式识别国家重点实验室）；SAI, UCAS（中国科学院大学人工智能学院）；ChaLearn；CVC, UB（巴塞罗那大学计算机视觉中心）；INAOE（国家天体物理、光学和电子学研究所）；CINVESTAV（研究与高级研究中心）；CAIR, HKISI, CAS（中国科学院香港创新研究院人工智能与机器人创新中心）；Baidu Research（百度研究院）
Ajian Liu, Chenxu Zhao, Zitong Yu, Anyang Su, Xing Liu, Zijian Kong, Jun Wan, Sergio Escalera, Hugo Jair Escalante, Zhen Lei, Guodong Guo
The threat of 3D masks to face recognition systems is increasingly serious and has been widely concerned by researchers. To facilitate the study of the algorithms, a large-scale High-Fidelity Mask dataset, namely CASIA-SURF HiFiMask (briefly HiFiMask) has been collected. Specifically, it consists of a total amount of 54, 600 videos which are recorded from 75 subjects with 225 realistic masks under 7 new kinds of sensors. Based on this dataset and Protocol 3 which evaluates both the discrimination and generalization ability of the algorithm under the open set scenarios, we organized a 3D High-Fidelity Mask Face Presentation Attack Detection Challenge to boost the research of 3D mask-based attack detection. It attracted 195 teams for the development phase with a total of 18 teams qualifying for the final round. All the results were verified and re-run by the organizing team, and the results were used for the final ranking. This paper presents an overview of the challenge, including the introduction of the dataset used, the definition of the protocol, the calculation of the evaluation criteria, and the summary and publication of the competition results. Finally, we focus on introducing and analyzing the top ranking algorithms, the conclusion summary, and the research ideas for mask attack detection provided by this competition.
Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形研究所）；TU Darmstadt（达姆施塔特工业大学）
Face masks have become one of the main methods for reducing the transmission of COVID-19. This makes face recognition (FR) a challenging task because masks hide several discriminative features of faces. Moreover, face presentation attack detection (PAD) is crucial to ensure the security of FR systems. In contrast to the growing number of masked FR studies, the impact of face masked attacks on PAD has not been explored. Therefore, we present novel attacks with real face masks placed on presentations and attacks with subjects wearing masks to reflect the current real-world situation. Furthermore…
School of Electronics and Information, Northwestern Polytechnical University（西北工业大学电子信息学院）；Department of Electrical and Electronic Engineering, University of Cagliari（卡利亚里大学电气与电子工程系）
Face presentation attacks have become a major threat to face recognition systems and many countermeasures have been proposed in the past decade. However, most of them are devoted to 2D face presentation attacks, rather than 3D face masks. Unlike the real face, the 3D face mask is usually made of resin materials and has a smooth surface, resulting in reflectance differences. So, we propose a novel detection method for 3D face mask presentation attack by modeling reflectance differences based on intrinsic image analysis. In the proposed method, the face image is first processed with intrinsic im…
3D mask face presentation attack detection (PAD) plays a vital role in securing face recognition systems from emergent 3D mask attacks. Recently, remote photoplethysmography (rPPG) has been developed as an intrinsic liveness clue for 3D mask PAD without relying on the mask appearance. However, the rPPG features for 3D mask PAD are still needed expert knowledge to design manually, which limits its further progress in the deep learning and big data era. In this letter, we propose a pure rPPG transformer (TransRPPG) framework for learning intrinsic liveness representation efficiently. At first, r…
MUST（澳门科技大学）；Mininglamp Academy of Sciences, Mininglamp Technology（明略科技集团明略科学院）；NLPR, CASIA, UCAS（中国科学院自动化研究所模式识别国家重点实验室，中国科学院大学）；CVC, UB（巴塞罗那大学计算机视觉中心）；Baidu Research（百度研究院）；Westlake University（西湖大学）；University of Oulu（奥卢大学）
Face presentation attack detection (PAD) is essential to secure face recognition systems primarily from high-fidelity mask attacks. Most existing 3D mask PAD benchmarks suffer from several drawbacks: 1) a limited number of mask identities, types of sensors, and a total number of videos; 2) low-fidelity quality of facial masks. Basic deep models and remote photoplethysmography (rPPG) methods achieved acceptable performance on these benchmarks but still far from the needs of practical scenarios. To bridge the gap to real-world applications, we introduce a largescale High-Fidelity Mask dataset, n…
Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形研究所）；TU Darmstadt（达姆施塔特工业大学）
Wearing a mask has proven to be one of the most effective ways to prevent the transmission of SARS-CoV-2 coronavirus. However, wearing a mask poses challenges for different face recognition tasks and raises concerns about the performance of masked face presentation detection (PAD). The main issues facing the mask face PAD are the wrongly classified bona fide masked faces and the wrongly classified partial attacks (covered by real masks). This work addresses these issues by proposing a method that considers partial attack labels to supervise the PAD model training, as well as regional weighted …
Nanyang Technological University（南洋理工大学）；SailYond Technology（赛扬科技）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院）
Face recognition technology has been widely used in daily interactive applications such as checking-in and mobile payment due to its convenience and high accuracy. However, its vulnerability to presentation attacks (PAs) limits its reliable use in ultra-secure applicational scenarios. A presentation attack is first defined in ISO standard as: a presentation to the biometric data capture subsystem with the goal of interfering with the operation of the biometric system. Specifically, PAs range from simple 2D print, replay and more sophisticated 3D masks and partial masks. To defend the face reco…
Wuhan University（武汉大学）；West Virginia University（西弗吉尼亚大学）
We have witnessed rapid advances in both face presentation attack models and presentation attack detection (PAD) in recent years. Compared to widely studied 2D face presentation attacks (e.g. printed photos and video replays), 3D face presentation attacks are more challenging because face recognition systems (FRS) is more easily confused by the 3D characteristics of materials similar to real faces. Existing 3D face spoofing databases, mostly based on 3D facial masks, are restricted to small data size and suffer from poor authenticity due to the difficulty and expense of mask production. In thi…
Compared to 2D face presentation attacks (e.g. printed photos and video replays), 3D type attacks are more challenging to face recognition systems (FRS) by presenting 3D characteristics or materials similar to real faces. Existing 3D face spoofing databases, however, mostly based on 3D masks, are restricted to small data size or poor authenticity due to the production difficulty and high cost. In this work, we introduce the first wax figure face database, WFFD, as one type of super-realistic 3D presentation attacks to spoof the FRS. This database consists of 2200 images with both real and wax …
Universidad Autonoma de Madrid（马德里自治大学）；European Commission - DG Joint Research Centre（欧盟委员会联合研究中心）
The main scope of this chapter is to serve as an introduction to face presentation attack detection, including key resources and advances in the field in the last few years. The next pages present the different presentation attacks that a face recognition system can confront, in which an attacker presents to the sensor, mainly a camera, a Presentation Attack Instrument (PAI), that is generally a photograph, a video, or a mask, to try to impersonate a genuine user. First, we make an introduction of the current status of face recognition, its level of deployment, and its challenges. In addition,…
Face recognition is a mainstream biometric authentication method. However, vulnerability to presentation attacks (a.k.a spoofing) limits its usability in unsupervised applications. Even though there are many methods available for tackling presentation attacks (PA), most of them fail to detect sophisticated attacks such as silicone masks. As the quality of presentation attack instruments improves over time, achieving reliable PA detection with visual spectra alone remains very challenging. We argue that analysis in multiple channels might help to address this issue. In this context, we propose …
Identity authentication is the process of verifying one's identity. There are several identity authentication methods, among which biometric authentication is of utmost importance. Facial recognition is a sort of biometric authentication with various applications, such as unlocking mobile phones and accessing bank accounts. However, presentation attacks pose the greatest threat to facial recognition. A presentation attack is an attempt to present a non-live face, such as a photo, video, mask, and makeup, to the camera. Presentation attack detection is a countermeasure that attempts to identify…
The non-intrusive nature and high accuracy of face recognition algorithms have led to their successful deployment across multiple applications ranging from border access to mobile unlocking and digital payments. However, their vulnerability against sophisticated and cost-effective presentation attack mediums raises essential questions regarding its reliability. In the literature, several presentation attack detection algorithms are presented; however, they are still far behind from reality. The major problem with existing work is the generalizability against multiple attacks both in the seen a…
Surveillance Face Presentation Attack Detection Challenge
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）；Instituto Nacional de Astrofísica, Óptica y Electrónica（国家天体物理、光学与电子研究所）；CAS（中国科学院）
Learning Knowledge-based Prompts for Robust 3D Mask Presentation Attack Detection
School of Computer Science, University of South China（南方大学计算机科学学院）；New Laboratory of Pattern Recognition, MAIS, CASIA（模式识别新实验室，MAIS，CASIA）；School of Intelligence Science and Technology, Nanjing University（智能科学与技术学院，南京大学）；Department of Computer Science and Engineering, University of California, Merced（加州大学默塞德分校计算机科学与工程系）；Department of Computer Science and Engineering, Yonsei University（延世大学计算机科学与工程系）
Introducing Nylon Face Mask Attacks: A Dataset for Evaluating Generalised Face Presentation Attack Detection
Indian Institute of Information and Technology (IIIT)（印度信息科技学院）；Norwegian University of Science and Technology (NTNU)（挪威科学技术大学）；School of Physical and Applied Sciences, Goa University（果阿大学物理与应用科学学院）
Friedrich Schiller University Jena（弗里德里希·席勒大学耶拿分校）
The human face has a high potential for biometric identification due to its many individual traits. At the same time, such identification is vulnerable to biometric copies. These presentation attacks pose a great challenge in unsupervised authentication settings. As a countermeasure, we propose a method that automatically analyzes the plausibility of facial behavior based on a sequence of 3D face scans. A compact feature representation measures facial behavior using the temporal curvature change. Finally, we train our method only on genuine faces in an anomaly detection scenario. Our method ca…
Manchester Metropolitan University（曼彻斯特城市大学）；Image Metrics Ltd（Image Metrics 公司）
Facial image inpainting, with high-fidelity preservation for image realism, is a very challenging task. This is due to the subtle texture in key facial features (component) that are not easily transferable. Many image inpainting techniques have been proposed with outstanding capabilities and high quantitative performances recorded. However, with facial inpainting, the features are more conspicuous and the visual quality of the blended inpainted regions are more important qualitatively. Based on these facts, we design a foreground-guided facial inpainting framework that can extract and generate…
Universidad Autonoma de Madrid（马德里自治大学）；Hochschule Darmstadt（达姆施塔特应用技术大学）
The increased need for unattended authentication in multiple scenarios has motivated a wide deployment of biometric systems in the last few years. This has in turn led to the disclosure of security concerns specifically related to biometric systems. Among them, Presentation Attacks (PAs, i.e., attempts to log into the system with a fake biometric characteristic or presentation attack instrument) pose a severe threat to the security of the system: any person could eventually fabricate or order a gummy finger or face mask to impersonate someone else. The biometrics community has thus made a cons…
Although two-stage Vector Quantized (VQ) generative models allow for synthesizing high-fidelity and high-resolution images, their quantization operator encodes similar patches within an image into the same index, resulting in a repeated artifact for similar adjacent regions using existing decoder architectures. To address this issue, we propose to incorporate the spatially conditional normalization to modulate the quantized vectors so as to insert spatially variant information to the embedded index maps, encouraging the decoder to generate more photorealistic images. Moreover, we use multichan…
The College of Management Academic Studies（管理学术学院）；The Weizmann Institute of Science（魏茨曼科学研究所）
Rich empirical evidence has shown that visual object recognition in the brain is fast and effortless, with relevant brain signals reported to start as early as 80 ms. Here we study the time trajectory of the recognition process at the level of minimal recognizable images (termed MIRC). These are images that can be recognized reliably, but in which a minute change of the image (reduction by either size or resolution) has a drastic effect on recognition. Subjects were assigned to one of nine exposure conditions: 200, 500, 1000, 2000 ms with or without masking, as well as unlimited time. The subj…
SARD: Segmentation-Aware Anomaly Synthesis via Region-Constrained Diffusion with Discriminative Mask Guidance
Global Institute of Future Technology（未来技术全球研究所）；Shanghai Jiao Tong University（上海交通大学）；Department of Intelligent Manufacturing（智能制造系）；Contemporary Amperex Technology Co.,Ltd（当代电子技术有限公司）
ModaFlow: Modality-Aware Flow Matching for High-Fidelity Virtual Try-On
South China University of Technology（华南理工大学）；Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）；Guangdong Provincial Key Laboratory of Multimodal Big Data Intelligent Analysis（广东省多模态大数据智能分析重点实验室）
Michigan State University（密歇根州立大学）；Qualcomm Technologies Inc.（高通技术公司）
Existing face relighting methods often struggle with two problems: maintaining the local facial details of the subject and accurately removing and synthesizing shadows in the relit image, especially hard shadows. We propose a novel deep face relighting method that addresses both problems. Our method learns to predict the ratio (quotient) image between a source image and the target image with the desired lighting, allowing us to relight the image while maintaining the local facial details. During training, our model also learns to accurately modify shadows by using estimated shadow masks to emp…
Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation
National University of Singapore（国立新加坡大学）；National University of Singapore Suzhou Research Institute（国立新加坡大学苏州研究院）；Wenzhou-Kean University（温州-凯恩大学）；Soochow University（苏州大学）；City University of Hong Kong（香港城市大学）
High-fidelity 3D Gaussian Inpainting: preserving multi-view consistency and photorealistic details
School of Information Science and Technology, Dalian Maritime University（信息科学与技术学院，大连海洋大学）；School of Science, Zhejiang Sci-Tech University（科学学院，浙江科技大学）
本文提出一种 3D Gaussian 修复框架，通过自动掩码精修和不确定性引导的区域级优化，利用稀疏修复视角重建完整 3D 场景，并在视觉质量与多视角一致性上超过现有方法。
City University of Hong Kong（香港城市大学）；Microsoft Cloud + AI（微软云与人工智能部门）
Image completion has made tremendous progress with convolutional neural networks (CNNs), because of their powerful texture modeling capacity. However, due to some inherent properties (e.g., local inductive prior, spatial-invariant kernels), CNNs do not perform well in understanding global structures or naturally support pluralistic completion. Recently, transformers demonstrate their power in modeling the long-term relationship and generating diverse results, but their computation complexity is quadratic to input length, thus hampering the application in processing high-resolution images. This…
Bridging global context interactions correctly is important for high-fidelity image completion with large masks. Previous methods attempting this via deep or large receptive field (RF) convolutions cannot escape from the dominance of nearby interactions, which may be inferior. In this paper, we propose to treat image completion as a directionless sequence-to-sequence prediction task, and deploy a transformer to directly capture long-range dependence in the encoder. Crucially, we employ a restrictive CNN with small and non-overlapping RF for weighted token representation, which allows the trans…
