The goal of the project "Facial Metrics for EES" is to develop, implement and publish an open source algorithm for the quality assessment of facial images (OFIQ) for face recognition, in particular for border control scenarios.1 In order to stimulate the harmonization of the requirements and practices applied for QA for facial images, the insights gained and algorithms developed in the project will be contributed to the current (2022) revision of the ISO/IEC 29794-5 standard. Furthermore, the implemented quality metrics and algorithms will consider the recommendations and requirements from other relevant standards, in particular ISO/IEC 19794-5:2011, ISO/IEC 29794-5:2010, ISO/IEC 39794-5:2019 and Version 5.2 of the BSI Technical Guideline TR-03121 Part 3 Volume 1. In order to establish an informed basis for the selection of quality metrics and the development of corresponding quality assessment algorithms, the state of the art of methods and algorithms (defining a metric), implementations and datasets for quality assessment for facial images is surveyed. For all relevant quality aspects, this document summarizes the requirements of the aforementioned standards, known results on their impact on face recognition performance, publicly available datasets, proposed methods and algorithms and open source software implementations.
Automated facial recognition system using deep learning for pain assessment in adults with cerebral palsy
Balearic ASPACE Foundation（巴利阿里ASPACE基金会）；University of the Balearic Islands（巴利阿里群岛大学）；Research Institute on Health Sciences (IUNICS)（健康科学研究所 (IUNICS)）；Health Research Institute of the Balearic Islands (IdISBa)（巴利阿里群岛健康研究所 (IdISBa)）；Federal University of ABC（ABC联邦大学）
Rapid development of artificial intelligence (AI) systems amplify many concerns in society. These AI algorithms inherit different biases from humans due to mysterious operational flow and because of that it is becoming adverse in usage. As a result, researchers have started to address the issue by investigating deeper in the direction towards Responsible and Explainable AI. Among variety of applications of AI, facial expression recognition might not be the most important one, yet is considered as a valuable part of human-AI interaction. Evolution of facial expression recognition from the featu…
Developing and Refining a Multifunctional Facial Recognition System for Older Adults with Cognitive Impairments: A Journey Towards Enhanced Quality of Life
Automated facial identification and facial expression recognition have been topics of active research over the past few decades. Facial and expression recognition find applications in human-computer interfaces, subject tracking, real-time security surveillance systems and social networking. Several holistic and geometric methods have been developed to identify faces and expressions using public and local facial image databases. In this work we present the evolution in facial image data sets and the methodologies for facial identification and recognition of expressions such as anger, sadness, h…
University of Essex（埃塞克斯大学）；Technological Educational Institute of Crete（克里特技术教育学院）；Aalborg University（奥尔堡大学）
Despite significant recent advances in the field of head pose estimation and facial expression recognition, raising the cognitive level when analysing human activity presents serious challenges to current concepts. Motivated by the need of generating comprehensible visual representations from different sets of data, we introduce a system capable of monitoring human activity through head pose and facial expression changes, utilising an affordable 3D sensing technology (Microsoft Kinect sensor). An approach build on discriminative random regression forests was selected in order to rapidly and ac…
Faculty of Electrical Engineering, University of Ljubljana（卢布尔雅那大学电气工程学院）；Faculty of Computer and Information Science, University of Ljubljana（卢布尔雅那大学计算机与信息科学学院）
Soft-biometric privacy-enhancing techniques represent machine learning methods that aim to: (i) mitigate privacy concerns associated with face recognition technology by suppressing selected soft-biometric attributes in facial images (e.g., gender, age, ethnicity) and (ii) make unsolicited extraction of sensitive personal information infeasible. Because such techniques are increasingly used in real-world applications, it is imperative to understand to what extent the privacy enhancement can be inverted and how much attribute information can be recovered from privacy-enhanced images. While these…
Sun Yat-sen University（中山大学）；The University of Hong Kong（香港大学）
Facial landmark localization plays a critical role in face recognition and analysis. In this paper, we propose a novel cascaded backbone-branches fully convolutional neural network~(BB-FCN) for rapidly and accurately localizing facial landmarks in unconstrained and cluttered settings. Our proposed BB-FCN generates facial landmark response maps directly from raw images without any preprocessing. BB-FCN follows a coarse-to-fine cascaded pipeline, which consists of a backbone network for roughly detecting the locations of all facial landmarks and one branch network for each type of detected landm…
We present a framework that uses GAN-augmented images to complement certain specific attributes, usually underrepresented, for machine learning model training. This allows us to improve inference quality over those attributes for the facial recognition tasks.
University of Southern California（南加州大学）；University of California San Francisco（加利福尼亚大学旧金山分校）
Facial action unit (FAU) intensities are popular descriptors for the analysis of facial behavior. However, FAUs are sparsely represented when only a few are activated at a time. In this study, we explore the possibility of representing the dynamics of facial expressions by adopting algorithms used for word representation in natural language processing. Specifically, we perform clustering on a large dataset of temporal facial expressions with 5.3M frames before applying the Global Vector representation (GloVe) algorithm to learn the embeddings of the facial clusters. We evaluate the usefulness …
The widespread deployment of surveillance cameras for facial recognition gives rise to many privacy concerns. This study proposes a privacy-friendly alternative to large scale facial recognition. While there are multiple techniques to preserve privacy, our work is based on the minimization principle which implies minimizing the amount of collected personal data. Instead of running facial recognition software on all video data, we propose to automatically extract a high quality snapshot of each detected person without revealing his or her identity. This snapshot is then encrypted and access is …
Universidad Autonoma de Madrid（马德里自治大学）；Veridas Digital Authentication Solutions（Veridas数字认证解决方案公司）
In this paper we develop FaceQvec, a software component for estimating the conformity of facial images with each of the points contemplated in the ISO/IEC 19794-5, a quality standard that defines general quality guidelines for face images that would make them acceptable or unacceptable for use in official documents such as passports or ID cards. This type of tool for quality assessment can help to improve the accuracy of face recognition, as well as to identify which factors are affecting the quality of a given face image and to take actions to eliminate or reduce those factors, e.g., with pos…
Accuracy and Fairness of Facial Recognition Technology in Low-Quality Police Images: An Experiment With Synthetic Faces
Department of Criminology, University of Pennsylvania（犯罪学系，宾夕法尼亚大学）；Department of Statistics and Data Science, Wharton School, University of Pennsylvania（统计与数据科学系，沃顿商学院，宾夕法尼亚大学）；Computer Science, School of Engineering & Applied Science, University of Pennsylvania（计算机科学，工程与应用科学学院，宾夕法尼亚大学）
TraMP-LLaMA: Generative Interpretability with Decoupled Instruction Tuning for Facial Expression Quality Assessment
School of Computer Science University of Bristol（布里斯托大学计算机科学学院）；Translational Health Sciences University of Bristol（布里斯托大学转化健康科学学院）；School of Computing and Communications Lancaster University（兰卡斯特大学计算与通信学院）
Current day pain assessment methods rely on patient self-report or by an observer like the Intensive Care Unit (ICU) nurses. Patient self-report is subjective to the individual and suffers due to poor recall. Pain assessment by manual observation is limited by the number of administrations per day and staff workload. Previous studies showed the feasibility of automatic pain assessment by detecting Facial Action Units (AUs). Pain is observed to be associated with certain facial action units (AUs). This method of pain assessment can overcome the pitfalls of present-day pain assessment techniques…
Sheep Facial Pain Assessment Under Weighted Graph Neural Networks
CISTER Research Center, Porto, Portugal（CISTER研究中心，葡萄牙 Porto）；Faculty of Engineering University of Porto, Portugal（葡萄牙 Porto大学工程学院）；Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStAL, F-59000, Lille, France（法国 Lille 大学，CNRS，Centrale Lille，UMR 9189 CRIStAL）；IMT Nord Europe, Institut Mines-Télécom, Univ. Lille, Centre for Digital Systems, F-59000 , Lille, France（法国 IMT Nord Europe，Institut Mines-Télécom，Univ. Lille，数字系统中心）
Sudan University of Science and Technology（苏丹科技大学）；Manchester Metropolitan University（曼彻斯特城市大学）
Facial wrinkle is one of the most prominent biological changes that accompanying the natural aging process. However, there are some external factors contributing to premature wrinkles development, such as sun exposure and smoking. Clinical studies have shown that heavy smoking causes premature wrinkles development. However, there is no computerised system that can automatically assess the facial wrinkles on the whole face. This study investigates the effect of smoking on facial wrinkling using a social habit face dataset and an automated computerised computer vision algorithm. The wrinkles pat…
Twins-PainViT: Towards a Modality-Agnostic Vision Transformer Framework for Multimodal Automatic Pain Assessment using Facial Videos and fNIRS
Computational Biomedicine Laboratory of the Foundation for Research and Technology (FORTH)（希腊国家研究与技术基金会计算生物医学实验室）；Department of Electrical & Computer Engineering Hellenic Mediterranean University（希腊地中海大学电气与计算机工程系）
This paper concerns with facial-based watchlist technology as a component of automated border control machines deployed in e-borders. The key task of the watchlist technology is to mitigate effects of mis-identification and impersonation. To address this problem, we developed a novel cost-based model of traveler risk assessment and proved its efficiency via intensive experiments using large-scale facial databases. The results of this study are applicable to any biometric modality to be used in watchlist technology.
Toronto Rehabilitation Institute – University Health Network（多伦多康复研究所 - 大学健康网络）；Massachusetts Eye and Ear Infirmary（麻省眼耳医院）；Harvard Medical School（哈佛医学院）；University of Toronto（多伦多大学）；Sunnybrook Research Institute（桑尼布鲁克研究所）；Royal Australasian College of Surgeons（澳大利亚皇家外科医师学院）；University of Sydney（悉尼大学）；Brasilia University（巴西利亚大学）
Importance: Machine learning (ML) approaches to facial landmark localization carry great clinical potential for quantitative assessment of facial function as they enable high-throughput automated quantification of relevant facial metrics from photographs. However, translation from research settings to clinical applications requires important improvements. Objective: To develop an ML algorithm for accurate facial landmarks localization in photographs of facial palsy patients, and use it as part of an automated computer-aided diagnosis system. Design, Setting, and Participants: Facial landmarks …
AI-driven Remote Facial Skin Hydration and TEWL Assessment from Selfie Images: A Systematic Solution
Rapid-Rich Object Search (ROSE) Lab, School of Electrical & Electronic Engineering, Nanyang Technological University (NTU), Singapore（快速-丰富对象搜索（ROSE）实验室，电气与电子工程学院，南洋理工大学（NTU），新加坡）；Research and Development, Beauty Care, P&G International Operations(SA), Singapore（研发，美容护理，P&G国际运营（南非），新加坡）
Automatic assessment and understanding of facial skin condition have several applications, including the early detection of underlying health problems, lifestyle and dietary treatment, skin-care product recommendation, etc. Selfies in the wild serve as an excellent data resource to democratize skin quality assessment, but suffer from several data collection challenges.The key to guaranteeing an accurate assessment is accurate detection of different skin features. We present an automatic facial skin feature detection method that works across a variety of skin tones and age groups for selfies in…
Indian Institute of Technology Guwahati（印度理工学院古瓦哈提分校）
Face image quality is an important factor in facial recognition systems as its verification and recognition accuracy is highly dependent on the quality of image presented. Rejecting low quality images can significantly increase the accuracy of any facial recognition system. In this project, a simple approach is presented to train a deep convolutional neural network to perform end-to-end face image quality assessment. The work is done in 2 stages : First, generation of quality score label and secondly, training a deep convolutional neural network in a supervised manner to predict quality score …
Read My Ears! Horse Ear Movement Detection for Equine Affective State Assessment
Visual Analysis and Perception Lab, Aalborg University（奥尔堡大学视觉分析与感知实验室）；Department of Veterinary Clinical Sciences, University of Copenhagen（哥本哈根大学兽医临床科学系）
Surveillance Facial Image Quality Assessment: A Multi-dimensional Dataset and Lightweight Model
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University（上海交通大学图像通信与网络工程研究所）；School of Communication and Electronic Engineering, East China Normal University（东华大学通信与电子工程学院）；School of Computer Science and Technology, East China Normal University（东华大学计算机科学与技术学院）；Shanghai AI Lab（上海人工智能实验室）
Pain in 3D: Generating Controllable Synthetic Faces for Automated Pain Assessment
Vector Institute（向量研究所）；KITE Research Institute（KITE研究机构）；University Health Network（大学健康网络）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Department of Medical Imaging, University of Toronto（多伦多大学医学成像系）；Institute of Biomedical Engineering, University of Toronto（多伦多大学生物医学工程研究所）
CASIA（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Facial images in surveillance or mobile scenarios often have large view-point variations in terms of pitch and yaw angles. These jointly occurred angle variations make face recognition challenging. Current public face databases mainly consider the case of yaw variations. In this paper, a new large-scale Multi-yaw Multi-pitch high-quality database is proposed for Facial Pose Analysis (M2FPA), including face frontalization, face rotation, facial pose estimation and pose-invariant face recognition. It contains 397,544 images of 229 subjects with yaw, pitch, attribute, illumination and accessory. …
In Biometrics, facial uniqueness is commonly inferred from impostor similarity scores. In this paper, we show that such uniqueness measures are highly unstable in the presence of image quality variations like pose, noise and blur. We also experimentally demonstrate the instability of a recently introduced impostor-based uniqueness measure of [Klare and Jain 2013] when subject to poor quality facial images.
Exploring Facial Biomarkers for Detecting Depression through Temporal Analysis of Action Units
Machine Learning and Data Analytics Lab, Department Artificial Intelligence in Biomedical Engineering (AIBE), Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（机器学习与数据分析实验室，人工智能生物医学工程部门（AIBE），弗里德里希-亚历山大-埃森纳赫大学（FAU））；Department of Clinical Psychology and Psychotherapy, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（临床心理学与心理治疗系，弗里德里希-亚历山大-埃森纳赫大学（FAU））；Chair of Visual Computing, Department of Computer Science, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU)（视觉计算教授，计算机科学系，弗里德里希-亚历山大-埃森纳赫大学（FAU））；Translational Digital Health Group, Institute of AI for Health, Helmholtz Zentrum München – German Research Center for Environmental Health（转化数字健康组，人工智能与健康研究所，海德堡中心——德国环境健康研究研究中心）
We present a single-image 3D face synthesis technique that can handle challenging facial expressions while recovering fine geometric details. Our technique employs expression analysis for proxy face geometry generation and combines supervised and unsupervised learning for facial detail synthesis. On proxy generation, we conduct emotion prediction to determine a new expression-informed proxy. On detail synthesis, we present a Deep Facial Detail Net (DFDN) based on Conditional Generative Adversarial Net (CGAN) that employs both geometry and appearance loss functions. For geometry, we capture 366…
SynPAIN: A Synthetic Dataset of Pain and Non-Pain Facial Expressions
KITE Research Institute, Toronto Rehabilitation Institute, University Health Network（KITE研究 institute、多伦多康复研究所、大学健康网络）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Institute of Biomedical Engineering, University of Toronto（多伦多大学生物医学工程研究所）；Vector Institute（向量研究所）；Department of Occupational Science and Occupational Therapy, University of Toronto（多伦多大学职业科学与职业治疗系）；University of Regina（Regina 多伦多大学）
While deep learning technologies are now capable of generating realistic images confusing humans, the research efforts are turning to the synthesis of images for more concrete and application-specific purposes. Facial image generation based on vocal characteristics from speech is one of such important yet challenging tasks. It is the key enabler to influential use cases of image generation, especially for business in public security and entertainment. Existing solutions to the problem of speech2face renders limited image quality and fails to preserve facial similarity due to the lack of qualit…
3D Weighted Geometric Graph Neural Networks for Sheep Facial Pain Assessment
CISTER Research Center（CISTER研究中心）；Faculty of Engineering University of Porto（波尔图大学工程学院）；Univ. Lille（里尔大学）；CNRS（法国国家科学研究中心）；Centrale Lille（里尔中央理工学院）；IMT Nord Europe（北欧电信学院）；Institut Mines-Télécom（法国国立矿业与电信学院）；Centre for Digital Systems（数字系统中心）
The face reenactment is a popular facial animation method where the person's identity is taken from the source image and the facial motion from the driving image. Recent works have demonstrated high quality results by combining the facial landmark based motion representations with the generative adversarial networks. These models perform best if the source and driving images depict the same person or if the facial structures are otherwise very similar. However, if the identity differs, the driving facial structures leak to the output distorting the reenactment result. We propose a novel Facial…
We propose a real time deep learning framework for video-based facial expression capture. Our process uses a high-end facial capture pipeline based on FACEGOOD to capture facial expression. We train a convolutional neural network to produce high-quality continuous blendshape weight output from video training. Since this facial capture is fully automated, our system can drastically reduce the amount of labor involved in the development of modern narrative-driven video games or films involving realistic digital doubles of actors and potentially hours of animated dialogue per character. We demons…
Nanjing University of Science and Technology（南京理工大学）；Tencent AI Lab（腾讯人工智能实验室）
We present a large-scale facial UV-texture dataset that contains over 50,000 high-quality texture UV-maps with even illuminations, neutral expressions, and cleaned facial regions, which are desired characteristics for rendering realistic 3D face models under different lighting conditions. The dataset is derived from a large-scale face image dataset namely FFHQ, with the help of our fully automatic and robust UV-texture production pipeline. Our pipeline utilizes the recent advances in StyleGAN-based facial image editing approaches to generate multi-view normalized face images from single-image …
Automatic pain intensity assessment has a high value in disease diagnosis applications. Inspired by the fact that many diseases and brain disorders can interrupt normal facial expression formation, we aim to develop a computational model for automatic pain intensity assessment from spontaneous and micro facial variations. For this purpose, we propose a 3D deep architecture for dynamic facial video representation. The proposed model is built by stacking several convolutional modules where each module encompasses a 3D convolution kernel with a fixed temporal depth, several parallel 3D convolutio…
