Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition
Center for Artificial Intelligence Research (CAiRE)（人工智能研究中心（CAiRE））；Department of Electronic and Computer Engineering（电子与计算机工程系）；The Hong Kong University of Science and Technology（香港科技大学）
Despite the recent achievements made in the multi-modal emotion recognition task, two problems still exist and have not been well investigated: 1) the relationship between different emotion categories are not utilized, which leads to sub-optimal performance; and 2) current models fail to cope well with low-resource emotions, especially for unseen emotions. In this paper, we propose a modality-transferable model with emotion embeddings to tackle the aforementioned issues. We use pre-trained word embeddings to represent emotion categories for textual data. Then, two mapping functions are learned to transfer these embeddings into visual and acoustic spaces. For each modality, the model calculates the representation distance between the input sequence and target emotions and makes predictions based on the distances. By doing so, our model can directly adapt to the unseen emotions in any modality since we have their pre-trained embeddings and modality mapping functions. Experiments show that our model achieves state-of-the-art performance on most of the emotion categories. In addition, our model also outperforms existing baselines in the zero-shot and few-shot scenarios for unseen emotions.
CMATH: Cross-Modality Augmented Transformer with Hierarchical Variational Distillation for Multimodal Emotion Recognition in Conversation
Chongqing University of Technology（重庆理工大学）；Fuzhou University（福州大学）；China Automotive Engineering Research Institute Co., Ltd.（中国汽车工程研究院股份有限公司）；Chongqing Changan Automobile Co., Ltd.（重庆长安汽车股份有限公司）
CFN-ESA: A Cross-Modal Fusion Network with Emotion-Shift Awareness for Dialogue Emotion Recognition
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology (HUST)（华中科技大学人工智能与自动化学院）；Institute of Artificial Intelligence, HUST（华中科技大学人工智能研究院）；Hubei Key Laboratory of Brain-inspired Intelligent Systems, HUST（华中科技大学湖北省脑启发智能系统重点实验室）；Key Laboratory of Image Processing and Intelligent Control (HUST), Ministry of Education（华中科技大学图像信息处理与智能控制教育部重点实验室）
UniMEEC: Towards Unified Multimodal Emotion Recognition and Emotion Cause
Peking University（北京大学）；King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；Arizona State University（亚利桑那州立大学）；The Hong Kong Polytechnic University（香港理工大学）；University of Copenhagen（哥本哈根大学）
Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics
University of Copenhagen（哥本哈根大学）
本文旨在测试多模态方法的音乐情感识别（MER）是否比单模态的高层次的歌曲特征和歌词。我们使用从Spotify API检索的11个歌曲特征，组合歌词特征，包括情感，TF-IDF和Anew，以预测Deezer情绪检测数据集（DMDD）（Delbouys et al.，2018），具有4种不同的回归模型。我们发现，在11个高级歌曲特征中，主要有5个对性能有贡献，多模态特征在预测效价时比单独的音频做得更好。我们使我们的代码可供使用。
Multi-modal Anchor Gated Transformer with Knowledge Distillation for Emotion Recognition in Conversation
School of Computer Science and Technology, China University of Mining and Technology（计算机科学与技术学院，中国矿业大学）；Mine Digitization Engineering Research Center of Ministry of Education, China University of Mining and Technology（教育部矿山数字化工程研究中心，中国矿业大学）
LMR-CBT: Learning Modality-fused Representations with CB-Transformer for Multimodal Emotion Recognition from Unaligned Multimodal Sequences
Beijing University of Posts and Telecommunications（北京邮电大学）；East China Normal University（华东师范大学）；Shanghai University of International Business and Economics（上海对外经贸大学）
EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation
Dept. of Electrical and Electronic Engineering, The Hong Kong Polytechnic University（香港理工大学电子与电气工程系）；Speech, Language, and Cognition Laboratory, The University of Hong Kong（香港大学语音、语言与认知实验室）
Emotion Across Speech and Faces: Shared Affective Mechanisms in Multimodal Foundation Models
Center for Language and Speech Processing (CLSP), Johns Hopkins University（约翰霍普金斯大学语言与语音处理中心）；Group on Language, Audio & Music (GLAM), Imperial College London（伦敦帝国理工学院语言、音频与音乐组）
Relational graph-driven differential denoising and diffusion attention fusion for multimodal conversation emotion recognition
College of Computer and Mathematics, Central South University of Forestry and Technology（中南林业科技大学计算机与数学学院）；Department of Computer Science, State University of New York, New Paltz（纽约州立大学新帕尔茨分校计算机科学系）
MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Peking University（北京大学）；Shandong Normal University（山东师范大学）；Renmin University of China（中国人民大学）；Ant Group（蚂蚁集团）；Nanyang Technological University（南洋理工大学）；University of Oulu（奥卢大学）
GSDNet: Revisiting Incomplete Multimodal-Diffusion from Graph Spectrum Perspective for Conversation Emotion Recognition
College of Computer and Mathematics, Central South University of Forestry and Technology（计算机与数学学院，中央南大学林业科技学院）；Department of Computer Science, Anhui Normal University（计算机科学系，安徽师范大学）；Future Technology Institute, South China University of Technology（未来技术研究院，华南理工大学）；Department of Computer Science, State University of New York（计算机科学系，纽约州立大学）
