3D Facial Matching by Spiral Convolutional Metric Learning and a Biometric Fusion-Net of Demographic Properties
KU Leuven（天主教鲁汶大学）；Imperial College London（伦敦帝国学院）；USI Lugano（卢加诺大学）；Indiana University-Purdue University-Indianapolis（印第安纳大学-普渡大学印第安纳波利斯联合分校）；Penn State University（宾夕法尼亚州立大学）；Twitter（推特）
Soha Sadat Mahdi, Nele Nauwelaers, Philip Joris, Giorgos Bouritsas, Shunwang Gong, Sergiy Bokhnyak, Susan Walsh, Mark D. Shriver, Michael Bronstein, Peter Claes, .
备注：
Accepted at International Conference on Pattern Recognition (ICPR 2020). Mahdi and Nauwelaers contributed equally and are ordered alphabetically
Face recognition is a widely accepted biometric verification tool, as the face contains a lot of information about the identity of a person. In this study, a 2-step neural-based pipeline is presented for matching 3D facial shape to multiple DNA-related properties (sex, age, BMI and genomic background). The first step consists of a triplet loss-based metric learner that compresses facial shape into a lower dimensional embedding while preserving information about the property of interest. Most studies in the field of metric learning have only focused on 2D Euclidean data. In this work, geometric deep learning is employed to learn directly from 3D facial meshes. To this end, spiral convolutions are used along with a novel mesh-sampling scheme that retains uniformly sampled 3D points at different levels of resolution. The second step is a multi-biometric fusion by a fully connected neural network. The network takes an ensemble of embeddings and property labels as input and returns genuine and imposter scores. Since embeddings are accepted as an input, there is no need to train classifiers for the different properties and available data can be used more efficiently. Results obtained by a 10-fold cross-validation for biometric verification show that combining multiple properties leads to stronger biometric systems. Furthermore, the proposed neural-based pipeline outperforms a linear baseline, which consists of principal component analysis, followed by classification with linear support vector machines and a Naive Bayes-based score-fuser.
This study demonstrates how facial biometrics, acquired using multi-spectral sensors, such as RGB, depth, and infrared, assist the data accumulation in the process of authorizing users of automated and semi-automated access systems. This data serves the purposes of person authentication, as well as facial temperature estimation. We utilize depth data taken using an inexpensive RGB-D sensor to find the head pose of a subject. This allows the selection of video frames containing a frontal-view head pose for face recognition and face temperature reading. Usage of the frontal-view frames improves …
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；L. D. College of Engineering（L. D.工程学院）；Ahmedabad University（艾哈迈达巴德大学）；Dosepack LLC（Dosepack有限责任公司）；Meditab Software Inc.（Meditab软件公司）
How can we improve the facial soft-biometric classification with help of the human visual system? This paper explores the use of saliency which is equivalent to the human visual system to classify Age, Gender and Facial Expression soft-biometric for facial images. Using the Deep Multi-level Network (ML-Net) [1] and off-the-shelf face detector [2], we propose our approach - SAF-BAGE, which first detects the face in the test image, increases the Bounding Box (B-Box) margin by 30%, finds the saliency map using ML-Net, with 30% reweighted ratio of saliency map, it multiplies with the input cropped…
National Institute of Standards and Technology（美国国家标准与技术研究院）
Traditionally, researchers in automatic face recognition and biometric technologies have focused on developing accurate algorithms. With this technology being integrated into operational systems, engineers and scientists are being asked, do these systems meet societal norms? The origin of this line of inquiry is `trust' of artificial intelligence (AI) systems. In this paper, we concentrate on adapting explainable AI to face recognition and biometrics, and we present four principles of explainable AI to face recognition and biometrics. The principles are illustrated by $\it{four}$ case studies,…
Imperial College London（帝国理工学院）；Middlesex University London（伦敦密德萨斯大学）
The progress we are currently witnessing in many computer vision applications, including automatic face analysis, would not be made possible without tremendous efforts in collecting and annotating large scale visual databases. To this end, we propose 4DFAB, a new large scale database of dynamic high-resolution 3D faces (over 1,800,000 3D meshes). 4DFAB contains recordings of 180 subjects captured in four different sessions spanning over a five-year period. It contains 4D videos of subjects displaying both spontaneous and posed facial behaviours. The database can be used for both face and facia…
Nokia Bell-Labs（诺基亚贝尔实验室）；Escuela Politecnica Superior, Universidad Autonoma de Madrid（马德里自治大学高等理工学院）
The role of soft biometrics to enhance person recognition systems in unconstrained scenarios has not been extensively studied. Here, we explore the utility of the following modalities: gender, ethnicity, age, glasses, beard, and moustache. We consider two assumptions: 1) manual estimation of soft biometrics and 2) automatic estimation from two commercial off-the-shelf systems (COTS). All experiments are reported using the labeled faces in the wild (LFW) database. First, we study the discrimination capabilities of soft biometrics standalone. Then, experiments are carried out fusing soft biometr…
Hochschule Darmstadt（达姆施塔特应用技术大学）；Fraunhofer Institute for Computer Graphics Research（弗劳恩霍夫计算机图形学研究所）；Norwegian University of Science and Technology（挪威科技大学）；University of Ljubljana（卢布尔雅那大学）
In the recent past, different researchers have proposed privacy-enhancing face recognition systems designed to conceal soft-biometric attributes at feature level. These works have reported impressive results, but generally did not consider specific attacks in their analysis of privacy protection. We introduce an attack on said schemes based on two observations: (1) highly similar facial representations usually originate from face images with similar soft-biometric attributes; (2) to achieve high recognition accuracy, robustness against intra-class variations within facial representations has t…
Universidad Autonoma de Madrid（马德里自治大学）；Max Planck Institute for Human Development（马克斯·普朗克人类发展研究所）
The most popular face recognition benchmarks assume a distribution of subjects without much attention to their demographic attributes. In this work, we perform a comprehensive discrimination-aware experimentation of deep learning-based face recognition. The main aim of this study is focused on a better understanding of the feature space generated by deep models, and the performance achieved over different demographic groups. We also propose a general formulation of algorithmic discrimination with application to face biometrics. The experiments are conducted over the new DiveFace database compo…
Hochschule Darmstadt（达姆施塔特应用科学大学）；Inria（法国国家信息与自动化研究所）；Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形研究所）；TU Darmstadt（达姆施塔特工业大学）
Systems incorporating biometric technologies have become ubiquitous in personal, commercial, and governmental identity management applications. Both cooperative (e.g. access control) and non-cooperative (e.g. surveillance and forensics) systems have benefited from biometrics. Such systems rely on the uniqueness of certain biological or behavioural characteristics of human beings, which enable for individuals to be reliably recognised using automated algorithms. Recently, however, there has been a wave of public and academic concerns regarding the existence of systemic bias in automated decisio…
Halmstad University（哈尔姆斯塔德大学）；University of Balearic Islands（巴利阿里群岛大学）
We address the use of selfie ocular images captured with smartphones to estimate age and gender. Partial face occlusion has become an issue due to the mandatory use of face masks. Also, the use of mobile devices has exploded, with the pandemic further accelerating the migration to digital services. However, state-of-the-art solutions in related tasks such as identity or expression recognition employ large Convolutional Neural Networks, whose use in mobile devices is infeasible due to hardware limitations and size restrictions of downloadable applications. To counteract this, we adapt two exist…
Recently, different researchers have found that the gallery composition of a face database can induce performance differentials to facial identification systems in which a probe image is compared against up to all stored reference images to reach a biometric decision. This negative effect is referred to as "watchlist imbalance effect". In this work, we present a method to theoretically estimate said effect for a biometric identification system given its verification performance across demographic groups and the composition of the used gallery. Further, we report results for identification expe…
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Research Center for AI Ethics and Safety, Beijing Academy of Artificial Intelligence（北京人工智能研究院人工智能伦理与安全研究中心）；Berggruen Research Center, Peking University（北京大学贝格伦研究中心）
Facial recognition is changing the way we live in and interact with our society. Here we discuss the two sides of facial recognition, summarizing potential risks and current concerns. We introduce current policies and regulations in different countries. Very importantly, we point out that the risks and concerns are not only from facial recognition, but also realistically very similar to other biometric recognition technology, including but not limited to gait recognition, iris recognition, fingerprint recognition, voice recognition, etc. To create a responsible future, we discuss possible tech…
Synthetically-generated audios and videos -- so-called deep fakes -- continue to capture the imagination of the computer-graphics and computer-vision communities. At the same time, the democratization of access to technology that can create sophisticated manipulated video of anybody saying anything continues to be of concern because of its power to disrupt democratic elections, commit small to large-scale fraud, fuel dis-information campaigns, and create non-consensual pornography. We describe a biometric-based forensic technique for detecting face-swap deep fakes. This technique combines a st…
Facial recognition technology and human raters can predict political orientation from images of expressionless faces even when controlling for demographics and self-presentation
Faculty of Electrical Engineering, University of Ljubljana（卢布尔雅那大学电气工程学院）；Faculty of Computer and Information Science, University of Ljubljana（卢布尔雅那大学计算机与信息科学学院）
Soft-biometric privacy-enhancing techniques represent machine learning methods that aim to: (i) mitigate privacy concerns associated with face recognition technology by suppressing selected soft-biometric attributes in facial images (e.g., gender, age, ethnicity) and (ii) make unsolicited extraction of sensitive personal information infeasible. Because such techniques are increasingly used in real-world applications, it is imperative to understand to what extent the privacy enhancement can be inverted and how much attribute information can be recovered from privacy-enhanced images. While these…
Universidad Autonoma de Madrid（马德里自治大学）；Veridas Digital Authentication Solutions（Veridas数字认证解决方案公司）
In this paper we develop FaceQvec, a software component for estimating the conformity of facial images with each of the points contemplated in the ISO/IEC 19794-5, a quality standard that defines general quality guidelines for face images that would make them acceptable or unacceptable for use in official documents such as passports or ID cards. This type of tool for quality assessment can help to improve the accuracy of face recognition, as well as to identify which factors are affecting the quality of a given face image and to take actions to eliminate or reduce those factors, e.g., with pos…
Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形学研究所）；Technical University of Darmstadt（达姆施塔特工业大学）
Current research on soft-biometrics showed that privacy-sensitive information can be deduced from biometric templates of an individual. Since for many applications, these templates are expected to be used for recognition purposes only, this raises major privacy issues. Previous works focused on supervised privacy-enhancing solutions that require privacy-sensitive information about individuals and limit their application to the suppression of single and pre-defined attributes. Consequently, they do not take into account attributes that are not considered in the training. In this work, we presen…
Hochschule Ansbach（安斯巴赫应用技术大学）；Hochschule Darmstadt（达姆施塔特应用技术大学）；EURECOM（欧洲通信学院）；Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形学研究所）
Since early 2020 the COVID-19 pandemic has had a considerable impact on many aspects of daily life. A range of different measures have been implemented worldwide to reduce the rate of new infections and to manage the pressure on national health services. A primary strategy has been to reduce gatherings and the potential for transmission through the prioritisation of remote working and education. Enhanced hand hygiene and the use of facial masks have decreased the spread of pathogens when gatherings are unavoidable. These particular measures present challenges for reliable biometric recognition…
CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61）；Cyber Security Cooperative Research Centre（网络安全合作研究中心）
Biometric data, such as face images, are often associated with sensitive information (e.g medical, financial, personal government records). Hence, a data breach in a system storing such information can have devastating consequences. Deep learning is widely utilized for face recognition (FR); however, such models are vulnerable to backdoor attacks executed by malicious parties. Backdoor attacks cause a model to misclassify a particular class as a target class during recognition. This vulnerability can allow adversaries to gain access to highly sensitive data protected by biometric authenticatio…
Hochschule Darmstadt（达姆施塔特应用技术大学）；University of Salzburg（萨尔茨堡大学）
Recently, concerns regarding potential biases in the underlying algorithms of many automated systems (including biometrics) have been raised. In this context, a biased algorithm produces statistically different outcomes for different groups of individuals based on certain (often protected by anti-discrimination legislation) attributes such as sex and age. While several preliminary studies investigating this matter for facial recognition algorithms do exist, said topic has not yet been addressed for vascular biometric characteristics. Accordingly, in this paper, several popular types of recogni…
Institute of Smart Cities（智慧城市研究所）；Public University of Navarre（纳瓦拉公立大学）
The increasing amount of applications of Artificial Intelligence (AI) has led researchers to study the social impact of these technologies and evaluate their fairness. Unfortunately, current fairness metrics are hard to apply in multi-class multi-demographic classification problems, such as Facial Expression Recognition (FER). We propose a new set of metrics to approach these problems. Of the three metrics proposed, two focus on the representational and stereotypical bias of the dataset, and the third one on the residual bias of the trained model. These metrics combined can potentially be used…
The performance of face recognition systems can be negatively impacted in the presence of masks and other types of facial coverings that have become prevalent due to the COVID-19 pandemic. In such cases, the periocular region of the human face becomes an important biometric cue. In this article, we present a detailed review of periocular biometrics. We first examine the various face and periocular techniques specially designed to recognize humans wearing a face mask. Then, we review different aspects of periocular biometrics: (a) the anatomical cues present in the periocular region useful for …
National Institute of Technology Karnataka（卡纳塔克邦国家理工学院）
Biometric systems based on Machine learning and Deep learning are being extensively used as authentication mechanisms in resource-constrained environments like smartphones and other small computing devices. These AI-powered facial recognition mechanisms have gained enormous popularity in recent years due to their transparent, contact-less and non-invasive nature. While they are effective to a large extent, there are ways to gain unauthorized access using photographs, masks, glasses, etc. In this paper, we propose an alternative authentication mechanism that uses both facial recognition and the…
In recent years, periocular recognition has been developed as a valuable biometric identification approach, especially in wild environments (for example, masked faces due to COVID-19 pandemic) where facial recognition may not be applicable. This paper presents a new deep periocular recognition framework called attribute-based deep periocular recognition (ADPR), which predicts soft biometrics and incorporates the prediction into a periocular recognition algorithm to determine identity from periocular images with high accuracy. We propose an end-to-end framework, which uses several shared convol…
Massachusetts Institute of Technology（麻省理工学院）；Sony AI（索尼人工智能研究院）
Harms resulting from the development and deployment of facial processing technologies (FPT) have been met with increasing controversy. Several states and cities in the U.S. have banned the use of facial recognition by law enforcement and governments, but FPT are still being developed and used in a wide variety of contexts where they primarily are regulated by state biometric information privacy laws. Among these laws, the 2008 Illinois Biometric Information Privacy Act (BIPA) has generated a significant amount of litigation. Yet, with most BIPA lawsuits reaching settlements before there have b…
Biometric technologies, especially face recognition, have become an essential part of identity management systems worldwide. In deployments of biometrics, secure storage of biometric information is necessary in order to protect the users' privacy. In this context, biometric cryptosystems are designed to meet key requirements of biometric information protection enabling a privacy-preserving storage and comparison of biometric data. This work investigates the application of a well-known biometric cryptosystem, i.e. the improved fuzzy vault scheme, to facial feature vectors extracted through deep…
Federal University of São Carlos（圣卡洛斯联邦大学）；São Paulo State University（圣保罗州立大学）
Biometrics emerged as a robust solution for security systems. However, given the dissemination of biometric applications, criminals are developing techniques to circumvent them by simulating physical or behavioral traits of legal users (spoofing attacks). Despite face being a promising characteristic due to its universality, acceptability and presence of cameras almost everywhere, face recognition systems are extremely vulnerable to such frauds since they can be easily fooled with common printed facial photographs. State-of-the-art approaches, based on Convolutional Neural Networks (CNNs), pre…
Cause of a rapid increase in technological development, increasing identity theft, consumer fraud, the threat to personal data is also increasing every day. Methods developed earlier to ensure personal the information from the thefts was not effective and safe. Biometrics were introduced when it was needed technology for more efficient security of personal information. Old-fashioned traditional approaches like Personal identification number( PIN), passwords, keys, login ID can be forgotten, stolen or lost. In biometric authentication system, user may not remember any passwords or carry any key…
School of Engineering, Universidad Autonoma de Madrid（马德里自治大学工程学院）；European Commission, JRC（欧盟委员会联合研究中心）
"The output of a computerised system can only be as accurate as the information entered into it." This rather trivial statement is the basis behind one of the driving concepts in biometric recognition: biometric quality. Quality is nowadays widely regarded as the number one factor responsible for the good or bad performance of automated biometric systems. It refers to the ability of a biometric sample to be used for recognition purposes and produce consistent, accurate, and reliable results. Such a subjective term is objectively estimated by the so-called biometric quality metrics. These algor…
Face biometrics are playing a key role in making modern smart city applications more secure and usable. Commonly, the recognition threshold of a face recognition system is adjusted based on the degree of security for the considered use case. The likelihood of a match can be for instance decreased by setting a high threshold in case of a payment transaction verification. Prior work in face recognition has unfortunately showed that error rates are usually higher for certain demographic groups. These disparities have hence brought into question the fairness of systems empowered with face biometri…
Deep Learning Models for Robust Facial Liveness Detection
Department of Theoretical and Applied Sciences, eCampus University（eCampus大学理论与应用科学系）；Department of Intelligent Software Systems and Technologies, School of Computer Science and Artificial Intelligence, V. N. Karazin Kharkiv National University（V. N. 卡拉津哈尔科夫国立大学计算机科学与人工智能学院智能软件系统与技术系）；Department of Political Sciences, Communication and International Relations, University of Macerata（马切拉塔大学政治学、传播与国际关系系）；Marche Polytechnic University（马尔凯理工大学）；Department Economics and Law, University of Macerata（马切拉塔大学经济与法律系）；Department of Information Engineering, Marche Polytechnic University（马尔凯理工大学信息工程系）
Over the past two decades, biometric recognition has exploded into a plethora of different applications around the globe. This proliferation can be attributed to the high levels of authentication accuracy and user convenience that biometric recognition systems afford end-users. However, in-spite of the success of biometric recognition systems, there are a number of outstanding problems and concerns pertaining to the various sub-modules of biometric recognition systems that create an element of mistrust in their use - both by the scientific community and also the public at large. Some of these …
On the Illusion of Gender Bias in Face Recognition: Explaining the Fairness Issue Through Non-demographic Attributes
Paderborn University（帕德博恩大学）；Technical University of Darmstadt（达姆施塔特技术大学）；University of Notre Dame（诺特丹大学）；Johannes Gutenberg University Mainz（美因茨约翰· Gutenberg大学）
Tandon School of Engineering, New York University（纽约大学坦登工程学院）；Center for Cyber Security, New York University Abu Dhabi（纽约大学阿布扎比分校网络安全中心）
Recent advances in Machine Learning (ML) have opened up new avenues for its extensive use in real-world applications. Facial recognition, specifically, is used from simple friend suggestions in social-media platforms to critical security applications for biometric validation in automated immigration at airports. Considering these scenarios, security vulnerabilities to such ML algorithms pose serious threats with severe outcomes. Recent work demonstrated that Deep Neural Networks (DNNs), typically used in facial recognition systems, are susceptible to backdoor attacks; in other words,the DNNs t…
Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形学研究所IGD）；TU Darmstadt（达姆施塔特工业大学）
Biases inherent in both data and algorithms make the fairness of widespread machine learning (ML)-based decision-making systems less than optimal. To improve the trustfulness of such ML decision systems, it is crucial to be aware of the inherent biases in these solutions and to make them more transparent to the public and developers. In this work, we aim at providing a set of explainability tool that analyse the difference in the face recognition models' behaviors when processing different demographic groups. We do that by leveraging higher-order statistical information based on activation map…
