To address the limitation in multimodal emotion recognition (MER) performance arising from inter-modal information fusion, we propose a novel MER framework based on multitask learning where fusion occurs after alignment, called Foal-Net. The framework is designed to enhance the effectiveness of modality fusion and includes two auxiliary tasks: audio-video emotion alignment (AVEL) and cross-modal emotion label matching (MEM). First, AVEL achieves alignment of emotional information in audio-video representations through contrastive learning. Then, a modal fusion network integrates the aligned features. Meanwhile, MEM assesses whether the emotions of the current sample pair are the same, providing assistance for modal information fusion and guiding the model to focus more on emotional information. The experimental results conducted on IEMOCAP corpus show that Foal-Net outperforms the state-of-the-art methods and emotion alignment is necessary before modal fusion.
DRKF: Decoupled Representations with Knowledge Fusion for Multimodal Emotion Recognition
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；School of Information and Software Engineering, University of Electronic Science and Technology of China（电子科技大学信息与软件工程学院）
A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learning
Central South University of Forestry and Technology（中部林业科技大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；State University of New York（纽约州立大学）
ASLSL: Adaptive shared latent structure learning with incomplete multi-modal physiological data for multi-dimensional emotional feature selection
School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China（信息科学与技术学院，北京理工大学，北京）；School of Artificial Intelligence, Anhui University, Beijing 100124, China（人工智能学院，安徽大学，北京）
Efficient Long-distance Latent Relation-aware Graph Neural Network for Multi-modal Emotion Recognition in Conversations
Central South University of Forestry and Technology（中南林业科技大学）；Changsha Medical University（长沙医学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
College of Computer and Mathematics, Central South University of Forestry and Technology（计算机与数学学院，中央南林业科技大学）；Department of Computer Science, State University of New York（计算机科学系，纽约州立大学）
Sync-TVA: A Graph-Attention Framework for Multimodal Emotion Recognition with Cross-Modal Fusion
James Watt School of Engineering, University of Glasgow（格拉斯哥大学詹姆斯·瓦特工程学院）；University of Bristol（布里斯托大学）；School of Engineering Mathematics and Technology, University of Bristol（布里斯托大学工程数学与技术学院）；School of Computing Science, University of Glasgow（格拉斯哥大学计算科学学院）；Department of Civil, Environmental & Geomatic Engineering, University College London (UCL)（伦敦大学学院（UCL）土木、环境与测绘工程系）
HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning
College of Artificial Intelligence（人工智能学院）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；School of Computer Science and Technology（计算机科学与技术学院）；Harbin Institute of Technology（哈尔滨工业大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳分校）
Multi-modal Attention for Speech Emotion Recognition
Institute of Data Science, NUS（新加坡国立大学数据科学研究所）；Graduate School for Integrative Sciences and Engineering, NUS（新加坡国立大学综合科学与工程研究生院）；National University of Singapore (NUS)（新加坡国立大学）；Osaka University（大阪大学）
Orthogonal Disentanglement with Projected Feature Alignment for Multimodal Emotion Recognition in Conversation
School of Computer Science Sichuan University（四川大学计算机学院）；Faculty of Computing Harbin Institute of Technology（哈尔滨工业大学计算学部）；College of Computer Science and Technology Harbin Engineering University（哈尔滨工程大学计算机科学与技术学院）
Multi-modal Speech Emotion Recognition via Feature Distribution Adaptation Network
School of Computer and Control Engineering, Yantai University（烟台大学计算机与控制工程学院）；Business School, Xuzhou University of Technology（徐州工程学院商学院）；College of Computer Science, Nankai University（南开大学计算机学院）；Southeast University（东南大学）
AV-EMO-Reasoning: Benchmarking Emotional Reasoning Capabilities in Omni-modal LLMS with Audio-visual Cues
UC Berkeley（加州大学伯克利分校）；South China University of Technology（华南理工大学）；Zhejiang University（浙江大学）；National Taiwan University（台湾大学）；University of Southern California（美国南加州大学）
Multi-Modality in Music: Predicting Emotion in Music from High-Level Audio Features and Lyrics
University of Copenhagen（哥本哈根大学）
本文旨在测试多模态方法的音乐情感识别（MER）是否比单模态的高层次的歌曲特征和歌词。我们使用从Spotify API检索的11个歌曲特征，组合歌词特征，包括情感，TF-IDF和Anew，以预测Deezer情绪检测数据集（DMDD）（Delbouys et al.，2018），具有4种不同的回归模型。我们发现，在11个高级歌曲特征中，主要有5个对性能有贡献，多模态特征在预测效价时比单独的音频做得更好。我们使我们的代码可供使用。
A cross-modal fusion network based on self-attention and residual structure for multimodal emotion recognition
Beijing University of Posts and Telecommunications（北京邮电大学）；East China Normal University（华东师范大学）；Shanghai University of International Business and Economics（上海对外经贸大学）
CMATH: Cross-Modality Augmented Transformer with Hierarchical Variational Distillation for Multimodal Emotion Recognition in Conversation
Chongqing University of Technology（重庆理工大学）；Fuzhou University（福州大学）；China Automotive Engineering Research Institute Co., Ltd.（中国汽车工程研究院股份有限公司）；Chongqing Changan Automobile Co., Ltd.（重庆长安汽车股份有限公司）
Emotion-Driven Personalized Recommendation for AI-Generated Content Using Multi-Modal Sentiment and Intent Analysis
University of Electronic Science and Technology of China（电子科技大学）；Columbia Business School, Columbia University（哥伦比亚大学哥伦比亚商学院）；New York University（纽约大学）
Angle-Optimized Partial Disentanglement for Multimodal Emotion Recognition in Conversation
Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算学部）；School of Astronautics, Beihang University（北京航空航天大学宇航学院）；Group of Intelligent Signal Processing (GISP), College of Computer Science and Technology, Harbin Engineering University（哈尔滨工程大学计算机科学与技术学院智能信号处理研究组）
