Qian Zheng, Ankur Purwar, Heng Zhao, Guang Liang Lim, Ling Li, Debasish Behera, Qian Wang, Min Tan, Rizhao Cai, Jennifer Werner, Dennis Sng, Maurice van Steense… 展开作者
Qian Zheng, Ankur Purwar, Heng Zhao, Guang Liang Lim, Ling Li, Debasish Behera, Qian Wang, Min Tan, Rizhao Cai, Jennifer Werner, Dennis Sng, Maurice van Steensel, Weisi Lin, Alex C Kot
备注：
Accepted by the conference of Electronic Imaging (EI) 2022
Automatic assessment and understanding of facial skin condition have several applications, including the early detection of underlying health problems, lifestyle and dietary treatment, skin-care product recommendation, etc. Selfies in the wild serve as an excellent data resource to democratize skin quality assessment, but suffer from several data collection challenges.The key to guaranteeing an accurate assessment is accurate detection of different skin features. We present an automatic facial skin feature detection method that works across a variety of skin tones and age groups for selfies in the wild. To be specific, we annotate the locations of acne, pigmentation, and wrinkle for selfie images with different skin tone colors, severity levels, and lighting conditions. The annotation is conducted in a two-phase scheme with the help of a dermatologist to train volunteers for annotation. We employ Unet++ as the network architecture for feature detection. This work shows that the two-phase annotation scheme can robustly detect the accurate locations of acne, pigmentation, and wrinkle for selfie images with different ethnicities, skin tone colors, severity levels, age groups, and lighting conditions.
AI-driven Remote Facial Skin Hydration and TEWL Assessment from Selfie Images: A Systematic Solution
Rapid-Rich Object Search (ROSE) Lab, School of Electrical & Electronic Engineering, Nanyang Technological University (NTU), Singapore（快速-丰富对象搜索（ROSE）实验室，电气与电子工程学院，南洋理工大学（NTU），新加坡）；Research and Development, Beauty Care, P&G International Operations(SA), Singapore（研发，美容护理，P&G国际运营（南非），新加坡）
Northeastern University（东北大学）；Beihang University（北京航空航天大学）；Microsoft Research Asia（微软亚洲研究院）；Meidaojia（美道家）
Modern approaches for semantic segmention usually pay too much attention to the accuracy of the model, and therefore it is strongly recommended to introduce cumbersome backbones, which brings heavy computation burden and memory footprint. To alleviate this problem, we propose an efficient segmentation method based on deep convolutional neural networks (DCNNs) for the task of hair and facial skin segmentation, which achieving remarkable trade-off between speed and performance on three benchmark datasets. As far as we know, the accuracy of skin tones classification is usually unsatisfactory due …
Key Laboratory for Biomedical Engineering of Ministry of Education（教育部生物医学工程重点实验室）；Zhejiang University of China（中国浙江大学）
In this paper, we present a new automatic diagnosis method of facial acne vulgaris based on convolutional neural network. This method is proposed to overcome the shortcoming of classification types in previous methods. The core of our method is to extract features of images based on convolutional neural network and achieve classification by classifier. We design a binary classifier of skin-and-non-skin to detect skin area and a seven-classifier to achieve the classification of facial acne vulgaris and healthy skin. In the experiment, we compared the effectiveness of our convolutional neural ne…
Semantic segmentation is a hot topic in computer vision where the most challenging tasks of object detection and recognition have been handling by the success of semantic segmentation approaches. We propose a concept of object-by-object learning technique to detect 11 types of facial skin lesions using semantic segmentation methods. Detecting individual skin lesion in a dense group is a challenging task, because of ambiguities in the appearance of the visual data. We observe that there exist co-occurrent visual relations between object classes (e.g., wrinkle and age spot, or papule and whitehe…
This paper is aimed at developing and combining different algorithms for face detection and face recognition to generate an efficient mechanism that can detect and recognize the facial regions of input image. For the detection of face from complex region, skin segmentation isolates the face-like regions in a complex image and following operations of morphology and template matching rejects false matches to extract facial region. For the recognition of the face, the image database is now converted into a database of facial segments. Hence, implementing the technique of Elastic Bunch Graph match…
Nanyang Technological University（南洋理工大学）；The Procter & Gamble Company（宝洁公司）；East China Normal University（华东师范大学）；Zhejiang University（浙江大学）；China-Singapore International Joint Research Institute（中新国际联合研究院）
Simulating the effects of skincare products on face is a potential new way to communicate the efficacy of skincare products in skin diagnostics and product recommendations. Furthermore, such simulations enable one to anticipate his/her skin conditions and better manage skin health. However, there is a lack of effective simulations today. In this paper, we propose the first simulation model to reveal facial pore changes after using skincare products. Our simulation pipeline consists of 2 steps: training data establishment and facial pore simulation. To establish training data, we collect face i…
In this paper, an efficient skin detection system is proposed. The algorithm is based on a very fast efficient pre-processing step utilizing the concept of ternary conversion in order to identify candidate windows and subsequently, a novel local two-stage diffusion method which has F-score accuracy of 0.5978 on SDD dataset. The pre-processing step has been proven to be useful to boost the speed of the system by eliminating 82% of an image in average. This is obtained by keeping the true positive rate above 98%. In addition, a novel segmentation algorithm is also designed to process candidate w…
Face detection is one of the challenging tasks in computer vision. Human face detection plays an essential role in the first stage of face processing applications such as face recognition, face tracking, image database management, etc. In these applications, face objects often come from an inconsequential part of images that contain variations, namely different illumination, poses, and occlusion. These variations can decrease face detection rate noticeably. Most existing face detection approaches are not accurate, as they have not been able to resolve unstructured images due to large appearanc…
Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Virtual facial avatars will play an increasingly important role in immersive communication, games and the metaverse, and it is therefore critical that they be inclusive. This requires accurate recovery of the appearance, represented by albedo, regardless of age, sex, or ethnicity. While significant progress has been made on estimating 3D facial geometry, albedo estimation has received less attention. The task is fundamentally ambiguous because the observed color is a function of albedo and lighting, both of which are unknown. We find that current methods are biased towards light skin tones due…
TrustSkin: A Fairness Pipeline for Trustworthy Facial Affect Analysis Across Skin Tone
Departamento de Física, Universidad de Tarapacá（物理系，塔拉帕卡大学）；Departamento de Ciencia de la Computación, Facultad de Ingeniería, Pontificia Universidad Católica de Chile（计算机科学系，工程学院，智利天主教大学）
Shinshu University（信州大学）；The University of Kitakyushu（北九州市立大学）；Tokyo Institute of Technology（东京技术大学）；Doshisha University（立命馆大学）；The University Institute of Technology La Rochelle（拉罗什大学技术学院）
本文提出了一种自动图像校正方法，用于人像照片，通过抑制背景颜色引起的肤色变化来提高面部肤色的一致性。在人像摄影中，由于光照环境（如从彩色背景墙反射的光线或相机 strobe 过曝）常导致肤色失真，若照片人工合成其他背景色，则这种颜色变化会更加明显，导致不自然的合成结果。在我们的框架中，首先大致提取面部区域并在颜色空间中校正肤色分布，然后在原始图像中对面部周围进行颜色和亮度校正，以实现适当的面部颜色平衡，不受亮度和背景颜色影响。与传统颜色校正算法不同，我们的最终结果通过带有引导图像的颜色校正过程获得。特别是，我们的引导图像过滤器在颜色校正中不需要像 He 等人最初提出的引导图像过滤器方法中所需的完美对齐的引导图像。实验结果表明，我们的方法在人像照片和自然场景照片上都比传统方法生成更自然的结果。我们还展示了自动年鉴风格照片生成作为另一种应用。
Selfie and cartoon are two popular artistic forms that are widely presented in our daily life. Despite the great progress in image translation/stylization, few techniques focus specifically on selfie cartoonization, since cartoon images usually contain artistic abstraction (e.g., large smoothing areas) and exaggeration (e.g., large/delicate eyebrows). In this paper, we address this problem by proposing a selfie cartoonization Generative Adversarial Network (scGAN), which mainly uses an attentive adversarial network (AAN) to emphasize specific facial regions and ignore low-level details. More s…
Similar to the majority of deep learning applications, diagnosing skin diseases using computer vision and deep learning often requires a large volume of data. However, obtaining sufficient data for particular types of facial skin conditions can be difficult due to privacy concerns. As a result, conditions like Rosacea are often understudied in computer-aided diagnosis. The limited availability of data for facial skin conditions has led to the investigation of alternative methods for computer-aided diagnosis. In recent years, Generative Adversarial Networks (GANs), mainly variants of StyleGANs,…
Pattern recognition problem rely upon the features inherent in the pattern of images. Face detection and recognition is one of the challenging research areas in the field of computer vision. In this paper, we present a method to identify skin pixels from still and video images using skin color. Face regions are identified from this skin pixel region. Facial features such as eyes, nose and mouth are then located. Faces are recognized from color images using an RBF based neural network. Unsupervised Cellular Automata with K means clustering algorithm is used to locate different facial elements. …
We worked with Nestle SHIELD (Skin Health, Innovation, Education, and Longevity Development, NSH) to develop a deep learning model that is able to assess acne severity from selfie images as accurate as dermatologists. The model was deployed as a mobile application, providing patients an easy way to assess and track the progress of their acne treatment. NSH acquired 4,700 selfie images for this study and recruited 11 internal dermatologists to label them in five categories: 1-Clear, 2- Almost Clear, 3-Mild, 4-Moderate, 5-Severe. Using OpenCV to detect facial landmarks we cut specific skin patch…
Shaheed Bhagat Singh College of Engineering & Technology（沙希德·巴加特·辛格工程技术学院）
Human Computer Interaction (HCI) is the biggest goal of computer vision researchers. Features form the different facial images are able to provide a very deep knowledge about the activities performed by the different facial movements. In this paper we presented a technique for feature extraction from various regions of interest with the help of Skin color segmentation technique, Thresholding, knowledge based technique for face recognition.
National Cheng Kung University（成功大学）；Umeå University（于默奥大学）
Facial feature tracking is a key component of imaging ballistocardiography (BCG) where accurate quantification of the displacement of facial keypoints is needed for good heart rate estimation. Skin feature tracking enables video-based quantification of motor degradation in Parkinson's disease. Traditional computer vision algorithms include Scale Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), and Lucas-Kanade method (LK). These have long represented the state-of-the-art in efficiency and accuracy but fail when common deformations, like affine local transformations or ill…
Human face perception is currently an active research area in the computer vision community. Skin detection is one of the most important and primary stages for this purpose. So far, many approaches are proposed to done this case. Near all of these methods have tried to find best match intensity distribution with skin pixels based on popular color spaces such as RGB, HSI or YCBCR. Results show that these methods cannot provide an accurate approach for every kind of skin. In this paper, an approach is proposed to solve this problem using a color probabilistic estimation technique. This approach …
RheinMain University of Applied Sciences（莱茵美因应用科学大学）；Bielefeld University（比勒费尔德大学）；University Medical Center of the Johannes Gutenberg University Mainz（美因茨约翰内斯·古腾堡大学医学中心）；Johannes Gutenberg University Mainz（美因茨约翰内斯·古腾堡大学）
In this paper, we present a method for automated estimation of a human face given a skull remain. The proposed method is based on three statistical models. A volumetric (tetrahedral) skull model encoding the variations of different skulls, a surface head model encoding the head variations, and a dense statistic of facial soft tissue thickness (FSTT). All data are automatically derived from computed tomography (CT) head scans and optical face scans. In order to obtain a proper dense FSTT statistic, we register a skull model to each skull extracted from a CT scan and determine the FSTT value for…
Arizona State University（亚利桑那州立大学）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
In this paper, we show that popular Generative Adversarial Networks (GANs) exacerbate biases along the axes of gender and skin tone when given a skewed distribution of face-shots. While practitioners celebrate synthetic data generation using GANs as an economical way to augment data for training data-hungry machine learning models, it is unclear whether they recognize the perils of such techniques when applied to real world datasets biased along latent dimensions. Specifically, we show that (1) traditional GANs further skew the distribution of a dataset consisting of engineering faculty headsh…
In recent times, there have been increasing accusations on artificial intelligence systems and algorithms of computer vision of possessing implicit biases. Even though these conversations are more prevalent now and systems are improving by performing extensive testing and broadening their horizon, biases still do exist. One such class of systems where bias is said to exist is facial recognition systems, where bias has been observed on the basis of gender, ethnicity, skin tone and other facial attributes. This is even more disturbing, given the fact that these systems are used in practically ev…
Encoder-decoder based architecture has been widely used in the generator of generative adversarial networks for facial manipulation. However, we observe that the current architecture fails to recover the input image color, rich facial details such as skin color or texture and introduces artifacts as well. In this paper, we present a novel method named SARGAN that addresses the above-mentioned limitations from three perspectives. First, we employed spatial attention-based residual block instead of vanilla residual blocks to properly capture the expression-related features to be changed while ke…
Hong Kong Baptist University（香港浸会大学）；Harbin Institute of Technology（哈尔滨工业大学）；The University of Hong Kong - Shenzhen Hospital（香港大学深圳医院）；Shenzhen Yiyuan Intelligent Technology Co., Ltd.（深圳怡园智能科技有限公司）
Skin disease is one of the most common types of human diseases, which may happen to everyone regardless of age, gender or race. Due to the high visual diversity, human diagnosis highly relies on personal experience; and there is a serious shortage of experienced dermatologists in many countries. To alleviate this problem, computer-aided diagnosis with state-of-the-art (SOTA) machine learning techniques would be a promising solution. In this paper, we aim at understanding the performance of convolutional neural network (CNN) based approaches. We first build two versions of skin disease datasets…
In this paper, we present a deep coupled framework to address the problem of matching sketch image against a gallery of mugshots. Face sketches have the essential in- formation about the spatial topology and geometric details of faces while missing some important facial attributes such as ethnicity, hair, eye, and skin color. We propose a cou- pled deep neural network architecture which utilizes facial attributes in order to improve the sketch-photo recognition performance. The proposed Attribute-Assisted Deep Con- volutional Neural Network (AADCNN) method exploits the facial attributes and le…
The presence of bias in deep models leads to unfair outcomes for certain demographic subgroups. Research in bias focuses primarily on facial recognition and attribute prediction with scarce emphasis on face detection. Existing studies consider face detection as binary classification into 'face' and 'non-face' classes. In this work, we investigate possible bias in the domain of face detection through facial region localization which is currently unexplored. Since facial region localization is an essential task for all face recognition pipelines, it is imperative to analyze the presence of such …
In this paper we propose an easiest approach for facial expression recognition. Here we are using concept of SVM for Expression Classification. Main problem is sub divided in three main modules. First one is Face detection in which we are using skin filter and Face segmentation. We are given more stress on feature Extraction. This method is effective enough for application where fast execution is required. Second, Facial Feature Extraction which is essential part for expression recognition. In this module we used Edge Projection Analysis. Finally extracted features vector is passed towards SVM…
Chaos in Motion: Unveiling Robustness in Remote Heart Rate Measurement through Brain-Inspired Skin Tracking
School of Information Science and Engineering, Lanzhou University（兰州大学信息科学与工程学院）；School of Electronics and Information Engineering, Lanzhou Jiaotong University（兰州交通大学电子与信息工程学院）；The First Hospital of Lanzhou University（兰州大学第一医院）；School of Artificial Intelligence, Nanjing University of Information Science and Technology（南京信息工程大学人工智能学院）
The rapid advancement of deep learning models that can generate and synthesis hyper-realistic videos known as Deepfakes and their ease of access to the general public have raised concern from all concerned bodies to their possible malicious intent use. Deep learning techniques can now generate faces, swap faces between two subjects in a video, alter facial expressions, change gender, and alter facial features, to list a few. These powerful video manipulation methods have potential use in many fields. However, they also pose a looming threat to everyone if used for harmful purposes such as iden…
Facial detection and analysis systems have been deployed by large companies and critiqued by scholars and activists for the past decade. Critiques that focus on system performance analyze disparity of the system's output, i.e., how frequently is a face detected for different Fitzpatrick skin types or perceived genders. However, we focus on the robustness of these system outputs under noisy natural perturbations. We present the first of its kind detailed benchmark of the robustness of three such systems: Amazon Rekognition, Microsoft Azure, and Google Cloud Platform. We use both standard and re…
Face signatures, including size, shape, texture, skin tone, eye color, appearance, and scars/marks, are widely used as discriminative, biometric information for access control. Despite recent advancements in facial recognition systems, presentation attacks on facial recognition systems have become increasingly sophisticated. The ability to detect presentation attacks or spoofing attempts is a pressing concern for the integrity, security, and trust of facial recognition systems. Multi-spectral imaging has been previously introduced as a way to improve presentation attack detection by utilizing …
Pak-Austria Institute of Applied Science and Technology（巴基斯坦-奥地利应用科学技术学院）；Superior College（高级学院）；Politecnico di Torino（都灵理工大学）
Race classification is a long-standing challenge in the field of face image analysis. The investigation of salient facial features is an important task to avoid processing all face parts. Face segmentation strongly benefits several face analysis tasks, including ethnicity and race classification. We propose a raceclassification algorithm using a prior face segmentation framework. A deep convolutional neural network (DCNN) was used to construct a face segmentation model. For training the DCNN, we label face images according to seven different classes, that is, nose, skin, hair, eyes, brows, bac…
University of Amsterdam（阿姆斯特丹大学）；DUniversum（3D宇宙）；Bilkent University（比尔肯特大学）；TNO（荷兰应用科学研究组织）
Lies and deception are common phenomena in society, both in our private and professional lives. However, humans are notoriously bad at accurate deception detection. Based on the literature, human accuracy of distinguishing between lies and truthful statements is 54% on average, in other words it is slightly better than a random guess. While people do not much care about this issue, in high-stakes situations such as interrogations for series crimes and for evaluating the testimonies in court cases, accurate deception detection methods are highly desirable. To achieve a reliable, covert, and non…
Detection and recognition of the facial images of people is an intricate problem which has garnered much attention during recent years due to its ever increasing applications in numerous fields. It continues to pose a challenge in finding a robust solution to it. Its scope extends to catering the security, commercial and law enforcement applications. Research for moreover a decade on this subject has brought about remarkable development with the modus operandi like human computer interaction, biometric analysis and content based coding of images, videos and surveillance. A trivial task for bra…
Karunya Institute of Technology and Sciences（卡鲁尼亚理工学院）；Manipal Institute of Technology, Manipal Academy of Higher Education（马尼帕尔高等教育学院马尼帕尔理工学院）
Around 40 percent of accidents related to driving on highways in India occur due to the driver falling asleep behind the steering wheel. Several types of research are ongoing to detect driver drowsiness but they suffer from the complexity and cost of the models. In this paper, SleepyWheels a revolutionary method that uses a lightweight neural network in conjunction with facial landmark identification is proposed to identify driver fatigue in real time. SleepyWheels is successful in a wide range of test scenarios, including the lack of facial characteristics while covering the eye or mouth, the…
