In emotion recognition, it is difficult to recognize human's emotional states using just a single modality. Besides, the annotation of physiological emotional data is particularly expensive. These two aspects make the building of effective emotion recognition model challenging. In this paper, we first build a multi-view deep generative model to simulate the generative process of multi-modality emotional data. By imposing a mixture of Gaussians assumption on the posterior approximation of the latent variables, our model can learn the shared deep representation from multiple modalities. To solve the labeled-data-scarcity problem, we further extend our multi-view model to semi-supervised learning scenario by casting the semi-supervised classification problem as a specialized missing data imputation task. Our semi-supervised multi-view deep generative framework can leverage both labeled and unlabeled data from multiple modalities, where the weight factor for each modality can be learned automatically. Compared with previous emotion recognition methods, our method is more robust and flexible. The experiments conducted on two real multi-modal emotion datasets have demonstrated the superiority of our framework over a number of competitors.
A Bayesian incorporated linear non-Gaussian acyclic model for multiple directed graph estimation to study brain emotion circuit development in adolescence
Tulane University（杜兰大学）；University of Nebraska Medical Center（内布拉斯加大学医学中心）；Mind Research Network（心智研究网络）；Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)（三机构神经影像与数据科学转化研究中心（TReNDS））；Georgia State University（佐治亚州立大学）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）
Bridging Modalities: Knowledge Distillation and Masked Training for Translating Multi-Modal Emotion Recognition to Uni-Modal, Speech-Only Emotion Recognition
HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning
College of Artificial Intelligence（人工智能学院）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；School of Computer Science and Technology（计算机科学与技术学院）；Harbin Institute of Technology（哈尔滨工业大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳分校）
ASLSL: Adaptive shared latent structure learning with incomplete multi-modal physiological data for multi-dimensional emotional feature selection
School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China（信息科学与技术学院，北京理工大学，北京）；School of Artificial Intelligence, Anhui University, Beijing 100124, China（人工智能学院，安徽大学，北京）
College of Computer and Mathematics, Central South University of Forestry and Technology（计算机与数学学院，中央南林业科技大学）；Department of Computer Science, State University of New York（计算机科学系，纽约州立大学）
A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learning
Central South University of Forestry and Technology（中部林业科技大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；State University of New York（纽约州立大学）
Efficient Long-distance Latent Relation-aware Graph Neural Network for Multi-modal Emotion Recognition in Conversations
Central South University of Forestry and Technology（中南林业科技大学）；Changsha Medical University（长沙医学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Sync-TVA: A Graph-Attention Framework for Multimodal Emotion Recognition with Cross-Modal Fusion
James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）；University of Bristol（布里斯托大学）；School of Engineering Mathematics and Technology, University of Bristol（布里斯托大学工程数学与技术学院）；School of Computing Science, University of Glasgow（格拉斯哥大学计算科学学院）；Department of Civil, Environmental & Geomatic Engineering, University College London (UCL)（伦敦大学学院（UCL）土木、环境与测绘工程系）
Emotion-aware Personalized Music Recommendation with a Heterogeneity-aware Deep Bayesian Network
School of Management, Hefei University of Technology（合肥工业大学管理学院）；Rawls College of Business, Texas Tech University（德克萨斯理工大学罗尔斯商学院）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）
Emotion-Driven Personalized Recommendation for AI-Generated Content Using Multi-Modal Sentiment and Intent Analysis
University of Electronic Science and Technology of China（电子科技大学）；Columbia Business School, Columbia University（哥伦比亚大学哥伦比亚商学院）；New York University（纽约大学）
Multi-Masked Querying Network for Robust Emotion Recognition from Incomplete Multi-Modal Physiological Signals
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, China（深圳先进技术研究院，中国科学院，深圳518055，中国）；Southern University of Science and Technology, Shenzhen 518055, China（南方科技大学，深圳518055，中国）
A Quantum Probability Driven Framework for Joint Multi-Modal Sarcasm, Sentiment and Emotion Analysis
Beijing Institute of Technology（北京理工大学）；The Open University（开放大学）；The Hong Kong Polytechnic University（香港理工大学）；Chongqing University（重庆大学）；China Mobile Communication Group Tianjin Co., Ltd.（中国移动通信集团天津有限公司）；Nanjing University（南京大学）；Zhengzhou University of Light Industry（郑州轻工业大学）
DRKF: Decoupled Representations with Knowledge Fusion for Multimodal Emotion Recognition
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件工程学院）
