Multimodal emotion recognition on CMU-MOSEI faces an extreme imbalance as Happy accounts for 65.9% of samples while three Ekman categories collectively represent under 7%, causing standard fusion models to maximize accuracy by ignoring minority emotions entirely. We present Affect-Diff, a Causal-Diffusion Bridge that addresses this through three jointly trained mechanisms: a NOTEARS-learned causal graph that re-weights modality contributions before fusion, a beta-VAE bottleneck for regularized latent compression, and a stop-gradiented 1D DDPM prior that structures the latent space against majority-class collapse. On 3,292 aligned CMU-MOSEI samples, Affect-Diff achieves validation balanced accuracy 0.384, an 18% relative improvement over the strongest baseline (TETFN: 0.324), while all evaluated baselines produce zero F1 on Fear, Disgust, and Surprise. Ablation studies confirm independent, non-redundant contributions from the diffusion prior (-24% without it) and causal graph (-13%). Notably, only the deterministic-encoder variant detects all six emotion classes, revealing KL regularization strength as a direct lever for minority-class sensitivity.
XEmoGPT: An Explainable Multimodal Emotion Recognition Framework with Cue-Level Perception and Reasoning
School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院);th Research Institute, China Electronics Technology Group Corporation(中国电子科技集团第五十四研究所)
GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Software Engineering Institute, Peking University(北京大学软件工程研究所);Department of Automation, Tsinghua University(清华大学自动化系);Beijing National Research Center for Information Science and Technology, Tsinghua University(清华大学北京信息科学与技术国家研究中心)
MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
The Chinese University of Hong Kong(香港中文大学);Tongyi Lab(通义实验室);SZTU(深圳技术大学);Peking University(北京大学);Tongji University(同济大学);CASIA(中国科学院自动化所);Tencent(腾讯);UCLA(加州大学洛杉矶分校);Westlake University(西湖大学);NTU(国立大学)
EmoAttack: Emotion-to-Image Diffusion Models for Emotional Backdoor Generation
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件工程学院);Tongji University(同济大学);College of Computer Science, Nankai University(南开大学计算机科学学院)
EmoTrans: A Benchmark for Understanding, Reasoning, and Predicting Emotion Transitions in Multimodal LLMs
Shenzhen University(深圳大学);Guangdong Laboratory of Artificial Intelligence(广东人工智能与数字经济实验室);Queen Mary University of London(女王学院伦敦大学);Technical University of Munich(慕尼黑技术大学);Imperial College London(伦敦帝国学院);Tongji University(同济大学)
Smile on the Face, Sadness in the Eyes: Bridging the Emotion Gap with a Multimodal Dataset of Eye and Facial Behaviors
School of Computer Science, China University of Geosciences (Wuhan)(中国地质大学(武汉)计算机科学学院);School of Software Engineering, Huazhong University of Science and Technology(华中科技大学软件工程学院);School of Computer Science, Wuhan University(武汉大学计算机科学学院);School of Computing, Engineering and Mathematical Sciences, La Trobe University(拉筹伯大学计算科学、工程与数学科学学院)
Generative Technology for Human Emotion Recognition: A Scope Review
Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ)(广东省人工智能与数字经济实验室(深圳));Tsinghua University(清华大学);The Hong Kong University of Science and Technology (GZ)(香港科技大学(广州));Beijing Normal University(北京师范大学);Shenzhen University(深圳大学);University of Electronic Science and Technology of China(电子科技大学)