Temporal Label Hierachical Network for Compound Emotion Recognition
School of Information Science and Engineering, Southeast University（东南大学信息科学与工程学院）；School of Biological Science and Medical Engineering, Southeast University（东南大学生物科学与医学工程学院）
The emotion recognition has attracted more attention in recent decades. Although significant progress has been made in the recognition technology of the seven basic emotions, existing methods are still hard to tackle compound emotion recognition that occurred commonly in practical application. This article introduces our achievements in the 7th Field Emotion Behavior Analysis (ABAW) competition. In the competition, we selected pre trained ResNet18 and Transformer, which have been widely validated, as the basic network framework. Considering the continuity of emotions over time, we propose a time pyramid structure network for frame level emotion prediction. Furthermore. At the same time, in order to address the lack of data in composite emotion recognition, we utilized fine-grained labels from the DFEW database to construct training data for emotion categories in competitions. Taking into account the characteristics of valence arousal of various complex emotions, we constructed a classification framework from coarse to fine in the label space.
MAFW: A Large-scale, Multi-modal, Compound Affective Database for Dynamic Facial Expression Recognition in the Wild
School of Computer Science, CUG University of Geosciences (Wuhan)（中国地质大学（武汉）计算机学院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）
Xiamen University（厦门大学）；University College London（伦敦大学学院）；Xiamen University of Technology（厦门理工学院）
Human emotions involve basic and compound facial expressions. However, current research on facial expression recognition (FER) mainly focuses on basic expressions, and thus fails to address the diversity of human emotions in practical scenarios. Meanwhile, existing work on compound FER relies heavily on abundant labeled compound expression training data, which are often laboriously collected under the professional instruction of psychology. In this paper, we study compound FER in the cross-domain few-shot learning setting, where only a few images of novel classes from the target domain are req…
MoEE: Mixture of Emotion Experts for Audio-Driven Portrait Animation
Zhejiang Lab（之江实验室）；Li Auto（理想汽车）；Harbin Institute of Technology（哈尔滨工业大学）；Zhejiang University（浙江大学）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）
This paper details the methodology and results of the EmotioNet challenge. This challenge is the first to test the ability of computer vision algorithms in the automatic analysis of a large number of images of facial expressions of emotion in the wild. The challenge was divided into two tracks. The first track tested the ability of current computer vision algorithms in the automatic detection of action units (AUs). Specifically, we tested the detection of 11 AUs. The second track tested the algorithms' ability to recognize emotion categories in images of facial expressions. Specifically, we te…
Automated emotion recognition in the wild from facial images remains a challenging problem. Although recent advances in Deep Learning have supposed a significant breakthrough in this topic, strong changes in pose, orientation and point of view severely harm current approaches. In addition, the acquisition of labeled datasets is costly, and current state-of-the-art deep learning algorithms cannot model all the aforementioned difficulties. In this paper, we propose to apply a multi-task learning loss function to share a common feature representation with other related tasks. Particularly we show…
Xidian University（西安电子科技大学）；JD Finance America Corporation（京东金融美国公司）；Peking University（北京大学）
By utilizing label distribution learning, a probability distribution is assigned for a facial image to express a compound emotion, which effectively improves the problem of label uncertainties and noises occurred in one-hot labels. In practice, it is observed that correlations among emotions are inherently different, such as surprised and happy emotions are more possibly synchronized than surprised and neutral. It indicates the correlation may be crucial for obtaining a reliable label distribution. Based on this, we propose a new method that amends the label distribution of each facial image b…
Revisiting Emotions Representation for Recognition in the Wild
São Paulo State Technological College (FATEC)（圣保罗州技术学院）；Media Integration and Communication Center (MICC) Department of Information Engineering, University of Florence（佛罗伦萨大学信息工程系媒体整合与通信中心）
A Survey on Facial Expression Recognition of Static and Dynamic Emotions
Academy for Engineering & Technology, Fudan University（复旦大学工程与技术学院）；College of Information Technology, Shanghai Ocean University（上海海洋大学信息技术学院）；School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）；School of Computer Science, Fudan University（复旦大学计算机科学学院）
AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels
Ritchie School of Engineering and Computer Science, University of Denver（丹佛大学里奇工程与计算机科学学院）；College of Arts, Humanities and Social Sciences, University of Denver（丹佛大学艺术、人文与社会科学学院）
Imperial College London（帝国理工学院）；FaceSoft Ltd（FaceSoft有限公司）；University of Oulu（奥卢大学）
Automatic facial behavior analysis has a long history of studies in the intersection of computer vision, physiology and psychology. However it is only recently, with the collection of large-scale datasets and powerful machine learning methods such as deep neural networks, that automatic facial behavior analysis started to thrive. Three of its iconic tasks are automatic recognition of basic expressions (e.g. happy, sad, surprised), estimation of continuous emotions (e.g., valence and arousal), and detection of facial action units (activations of e.g. upper/inner eyebrows, nose wrinkles). Up unt…
