Facial expressions are an integral part of human cognition and communication, and can be applied in various real life applications. A vital precursor to accurate expression recognition is feature extraction. In this paper, we propose SenTion: A framework for sensing facial expressions. We propose a novel person independent and scale invariant method of extracting Inter Vector Angles (IVA) as geometric features, which proves to be robust and reliable across databases. SenTion employs a novel framework of combining geometric (IVA's) and appearance based features (Histogram of Gradients) to create a hybrid model, that achieves state of the art recognition accuracy. We evaluate the performance of SenTion on two famous face expression data set, namely: CK+ and JAFFE; and subsequently evaluate the viability of facial expression systems by a user study. Extensive experiments showed that SenTion framework yielded dramatic improvements in facial expression recognition and could be employed in real-world applications with low resolution imaging and minimal computational resources in real-time, achieving 15-18 fps on a 2.4 GHz CPU with no GPU.
University of Essex（埃塞克斯大学）；Technological Educational Institute of Crete（克里特技术教育学院）；Aalborg University（奥尔堡大学）
Despite significant recent advances in the field of head pose estimation and facial expression recognition, raising the cognitive level when analysing human activity presents serious challenges to current concepts. Motivated by the need of generating comprehensible visual representations from different sets of data, we introduce a system capable of monitoring human activity through head pose and facial expression changes, utilising an affordable 3D sensing technology (Microsoft Kinect sensor). An approach build on discriminative random regression forests was selected in order to rapidly and ac…
EventFace: Event-Based Face Recognition via Structure-Driven Spatiotemporal Modeling
State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology（光电信息获取与防护技术国家重点实验室）；Anhui Provincial Key Laboratory of Secure Artificial Intelligence（安徽省安全人工智能重点实验室）；Anhui Provincial International Joint Research Center for Advanced Technology in Medical Imaging（安徽省医学影像先进技术国际联合研究中心）；School of Artificial Intelligence, Anhui University（安徽大学人工智能学院）；Computer Vision Laboratory, University of Sassari（萨萨里大学计算机视觉实验室）
Currently, many critical care indices are repetitively assessed and recorded by overburdened nurses, e.g. physical function or facial pain expressions of nonverbal patients. In addition, many essential information on patients and their environment are not captured at all, or are captured in a non-granular manner, e.g. sleep disturbance factors such as bright light, loud background noise, or excessive visitations. In this pilot study, we examined the feasibility of using pervasive sensing technology and artificial intelligence for autonomous and granular monitoring of critically ill patients an…
With recent advances in RGB-D sensing technologies as well as improvements in machine learning and fusion techniques, RGB-D facial recognition has become an active area of research. A novel attention aware method is proposed to fuse two image modalities, RGB and depth, for enhanced RGB-D facial recognition. The proposed method first extracts features from both modalities using a convolutional feature extractor. These features are then fused using a two-layer attention mechanism. The first layer focuses on the fused feature maps generated by the feature extractor, exploiting the relationship be…
Georgia Institute of Technology（佐治亚理工学院）；Google（谷歌）
One of the main challenges of social interaction in virtual reality settings is that head-mounted displays occlude a large portion of the face, blocking facial expressions and thereby restricting social engagement cues among users. Hence, auxiliary means of sensing and conveying these expressions are needed. We present an algorithm to automatically infer expressions by analyzing only a partially occluded face while the user is engaged in a virtual reality experience. Specifically, we show that images of the user's eyes captured from an IR gaze-tracking camera within a VR headset are sufficient…
Fast L1-Minimization Algorithms For Robust Face Recognition
University of California, Berkeley（加州大学伯克利分校）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；IBM Research（IBM研究院）；Microsoft Research Asia（微软亚洲研究院）
National Laboratory of Pattern Recognition, CASIA（中国科学院自动化研究所模式识别国家重点实验室）；Center for Research on Intelligent Perception and Computing, CASIA（中国科学院自动化研究所智能感知与计算研究中心）；Center for Excellence in Brain Science and Intelligence Technology, CAS（中国科学院脑科学与智能技术卓越创新中心）；University of Chinese Academy of Sciences（中国科学院大学）
Heterogeneous face recognition (HFR) aims to match facial images acquired from different sensing modalities with mission-critical applications in forensics, security and commercial sectors. However, HFR is a much more challenging problem than traditional face recognition because of large intra-class variations of heterogeneous face images and limited training samples of cross-modality face image pairs. This paper proposes a novel approach namely Wasserstein CNN (convolutional neural networks, or WCNN for short) to learn invariant features between near-infrared and visual face images (i.e. NIR-…
Towards Emotion Recognition with 3D Pointclouds Obtained from Facial Expression Images
Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）；University of Antwerp（安特卫普大学）；imec；i2CAT Foundation（i2CAT基金会）；University of Zagreb（萨格勒布大学）；NEC Labs Europe GmbH（NEC欧洲实验室有限公司）；ICREA
University of Washington（华盛顿大学）；Microsoft Research（微软研究院）
Numerous real-world applications have been driven by the recent algorithmic advancement of artificial intelligence (AI). Healthcare is no exception and AI technologies have great potential to revolutionize the industry. Non-contact camera-based physiological sensing, including remote photoplethysmography (rPPG), is a set of imaging methods that leverages ordinary RGB cameras (e.g., webcam or smartphone camera) to capture subtle changes in electromagnetic radiation (e.g., light) reflected by the body caused by physiological processes. RGB camera-based systems not only have the ability to measur…
Light field (LF) cameras provide rich spatio-angular visual representations by sensing the visual scene from multiple perspectives and have recently emerged as a promising technology to boost the performance of human-machine systems such as biometrics and affective computing. Despite the significant success of LF representation for constrained facial image analysis, this technology has never been used for face and expression recognition in the wild. In this context, this paper proposes a new deep face and expression recognition solution, called CapsField, based on a convolutional neural networ…
Smile on the Face, Sadness in the Eyes: Bridging the Emotion Gap with a Multimodal Dataset of Eye and Facial Behaviors
School of Computer Science, China University of Geosciences (Wuhan)（中国地质大学（武汉）计算机科学学院）；School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件工程学院）；School of Computer Science, Wuhan University（武汉大学计算机科学学院）；School of Computing, Engineering and Mathematical Sciences, La Trobe University（拉筹伯大学计算科学、工程与数学科学学院）
Affect (emotion) recognition has gained significant attention from researchers in the past decade. Emotion-aware computer systems and devices have many applications ranging from interactive robots, intelligent online tutor to emotion based navigation assistant. In this research data from multiple modalities such as face, head, hand, body and speech was utilized for affect recognition. The research used color and depth sensing device such as Kinect for facial feature extraction and tracking human body joints. Temporal features across multiple frames were used for affect recognition. Event drive…
Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach
Universitat de Barcelona（巴塞罗那大学）；Computer Vision Center（计算机视觉中心）；Universidad del País Vasco UPV-EHU（巴斯克大学）；SAMOVAR；Telecom SudParis（巴黎高等电信学院）；Institut Polytechnique de Paris（巴黎综合理工学院）；Università degli Studi della Campania L. Vanvitelli（坎帕尼亚大学）；Universidad Internacional de La Rioja（拉里奥哈国际大学）；Osatek；Oslo University Hospital（奥斯陆大学医院）
