Emotion Recognition in Conversations (ERC) presents unique challenges, requiring models to capture the temporal flow of multi-turn dialogues and to effectively integrate cues from multiple modalities. We propose Mixture of Speech-Text Experts for Recognition of Emotions (MiSTER-E), a modular Mixture-of-Experts (MoE) framework designed to decouple two core challenges in ERC: modality-specific context modeling and multimodal information fusion. MiSTER-E leverages large language models (LLMs) fine-tuned for both speech and text to provide rich utterance-level embeddings, which are then enhanced through a convolutional-recurrent context modeling layer. The system integrates predictions from three experts-speech-only, text-only, and cross-modal-using a learned gating mechanism that dynamically weighs their outputs. To further encourage consistency and alignment across modalities, we introduce a supervised contrastive loss between paired speech-text representations and a KL-divergence-based regulariza-tion across expert predictions. Importantly, MiSTER-E does not rely on speaker identity at any stage. Experiments on three benchmark datasets-IEMOCAP, MELD, and MOSI-show that our proposal achieves 70.9%, 69.5%, and 87.9% weighted F1-scores respectively, outperforming several baseline speech-text ERC systems. We also provide various ablations to highlight the contributions made in the proposed approach.
Emotion Transfer with Enhanced Prototype for Unseen Emotion Recognition in Conversation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);Beihang University(北航);National University of Defense Technology(国防科技大学);Shantou University(汕头大学);University of Illinois at Chicago(伊利诺伊大学芝加哥分校)
LaERC-S: Improving LLM-based Emotion Recognition in Conversation with Speaker Characteristics
School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院);School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)计算机科学与技术学院)
Dialogue Quality and Emotion Annotations for Customer Support Conversations
INESC-ID(INESC-ID(葡萄牙国家系统与计算机工程研究所));Instituto Superior Técnico, University of Lisbon(里斯本大学高等技术学院);Faculdade de Letras, University of Lisbon(里斯本大学文学院);Unbabel(Unbabel公司);Carnegie Mellon University(卡内基梅隆大学);Phrase(Phrase公司)
InstructERC: Reforming Emotion Recognition in Conversation with Multi-task Retrieval-Augmented Large Language Models
School of Artificial Intelligence and Automation, HUST(华中科技大学人工智能与自动化学院);School of Artificial Intelligence, BUPT(北京邮电大学人工智能学院);Sino-French Engineer School, Beihang University(北京航空航天大学中法工程师学院);Meituan Inc.(美团公司)
DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversations
The Hong Kong Polytechnic University(香港理工大学);Zhengzhou University of Light Industry(郑州轻工业大学);Hebei University of Technology(河北工业大学);Halmstad University(哈尔姆斯塔德大学);Copenhagen University(哥本哈根大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳))
Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling
Inner Mongolian University(内蒙古大学);ByteDance(字节跳动);Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));National University of Singapore(新加坡国立大学)
Emotion-Anchored Contrastive Learning Framework for Emotion Recognition in Conversation
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);School of Artificial Intelligence, Nanjing University(南京大学人工智能学院)
Facilitating Multi-turn Emotional Support Conversation with Positive Emotion Elicitation: A Reinforcement Learning Approach
Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心);Tsinghua University(清华大学);College of Intelligence and Computing, Tianjin University(天津大学智能与计算学部);Tianjin University(天津大学)
MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations
National University of Singapore(新加坡国立大学);Instituto Politécnico Nacional(墨西哥国立理工学院);Nanyang Technological University(南洋理工大学);University of Michigan(密歇根大学)
Emotion Eliciting Machine: Emotion Eliciting Conversation Generation based on Dual Generator
Huawei Poisson Lab.(华为 Poisson 实验室);Université de Montréal(蒙特利尔大学);Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院)
Speaker-Guided Encoder-Decoder Framework for Emotion Recognition in Conversation
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
DialogueRNN: An Attentive RNN for Emotion Detection in Conversations
National University of Singapore(新加坡国立大学);University of Michigan(密歇根大学);Instituto Politécnico Nacional(国立理工学院);Nanyang Technological University(南洋理工大学)
Conversational Emotion Analysis via Attention Mechanisms
CAS Center for Excellence in Brain Science and Intelligence Technology(中国科学院脑科学与智能技术卓越创新中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Attention-based Modeling for Emotion Detection and Classification in Textual Conversations
LIRMM UMR 5506;CNRS(法国国家科学研究中心);University of Montpellier(蒙彼利埃大学);IUT de Béziers(贝济耶大学技术学院);AMIS;Paul Valery University - Montpellier 3(保罗·瓦莱里大学-蒙彼利埃第三大学)