Automatic speech emotion recognition (SER) by a computer is a critical component for more natural human-machine interaction. As in human-human interaction, the capability to perceive emotion correctly is essential to take further steps in a particular situation. One issue in SER is whether it is necessary to combine acoustic features with other data such as facial expressions, text, and motion capture. This research proposes to combine acoustic and text information by applying a late-fusion approach consisting of two steps. First, acoustic and text features are trained separately in deep learning systems. Second, the prediction results from the deep learning systems are fed into a support vector machine (SVM) to predict the final regression score. Furthermore, the task in this research is dimensional emotion modeling because it can enable a deeper analysis of affective states. Experimental results show that this two-stage, late-fusion approach, obtains higher performance than that of any one-stage processing, with a linear correlation from one-stage to two-stage processing. This late-fusion approach improves previous early fusion results measured in concordance correlation coefficients score.
Speech Emotion Recognition Using MFCC Features and LSTM-Based Deep Learning Model
Department of Electrical Electronics and Telecommunications Engineering, Bells University of Technology(贝尔大学技术学院电子与电信工程系);New Horizons Bells University of Technology(新希望贝尔大学技术学院);Department of Computer Engineering, Bells University of Technology(贝尔大学技术学院计算机工程系);Department of Computer and Information Sciences, Towson University(托逊大学计算机与信息科学系)
Adapting General Disentanglement-Based Speaker Anonymization for Enhanced Emotion Preservation
Singapore Institute of Technology(新加坡理工学院);Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所语音声学与内容理解重点实验室);National Institute of Informatics(日本国立信息研究所);Inria, Centre Inria de l’Université de Lorraine(法国国家信息与自动化技术研究院(Inria)卢瓦尔大学中心)
Audio-Guided Fusion Techniques for Multimodal Emotion Analysis
Beijing University of Posts and Telecommunications(北京邮电大学);State Key Laboratory of Networking and Switching Technology(网络与交换技术国家重点实验室);School of Cyberspace Security(网络空间安全学院)
Robust Audio-Visual Target Speaker Extraction with Emotion-Aware Multiple Enrollment Fusion
School of Computer Science, Wuhan University, Wuhan, China(1 武汉大学计算机学院,武汉,中国);School of Artificial Intelligence, The Chinese University of Hong Kong, Shenzhen, China(2 香港中文大学(深圳)人工智能学院,中国);School of Artificial Intelligence, Wuhan University, Wuhan, China(3 武汉大学人工智能学院,武汉,中国);School of Cyber Science and Engineering, Wuhan University, Wuhan, China(4 武汉大学网络科学与工程学院,武汉,中国);Digital Innovation Research Center, Duke Kunshan University, Kunshan, China(5 香港中文大学(深圳)数字创新研究中心,中国);AI Center, OPPO, Beijing, China(6 OPPO人工智能中心,北京,中国)
LMR-CBT: Learning Modality-fused Representations with CB-Transformer for Multimodal Emotion Recognition from Unaligned Multimodal Sequences
Beijing University of Posts and Telecommunications(北京邮电大学);East China Normal University(华东师范大学);Shanghai University of International Business and Economics(上海对外经贸大学)
GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition
College of Computer and Mathematics, Central South University of Forestry and Technology(计算机与数学学院,中央南大学林业科技学院);Department of Computer Science, Anhui Normal University(计算机科学系,安徽师范大学);Future Technology Institute, South China University of Technology(未来技术研究院,华南理工大学);Department of Computer Science, State University of New York(计算机科学系,纽约州立大学)
College of Computer and Mathematics, Central South University of Forestry and Technology(计算机与数学学院,中央南林业科技大学);Department of Computer Science, State University of New York(计算机科学系,纽约州立大学)