Current Multimodal Affective Computing (MAC) systems heavily rely on the completeness of multiple modalities to accurately understand human's affective state. However, in real-world scenarios, the availability of modality data is often dynamic and uncertain, leading to substantial performance fluctuations due to the distribution shifts and semantic deficiencies of the incomplete multimodal inputs. Known as the missing modality issue, this challenge poses a critical barrier to the robustness and practical deployment of MAC models. To systematically quantify this issue, we introduce \textbf{MissMAC-Bench}, a comprehensive benchmark designed to establish fair and unified evaluation standards from the perspective of cross-modal synergy. Two guiding principles are proposed, including no missing prior during training, and one single model capable of handling both complete and incomplete modality scenarios, thereby ensuring better generalization. Moreover, to bridge the gap between academic research and real-world applications, our benchmark integrates evaluation protocols with both fixed and random missing patterns at the dataset and instance levels. Extensive experiments conducted on 3 widely-used language models across 4 datasets validate the effectiveness of diverse MAC approaches in tackling the missing modality issue. Our benchmark provides a solid foundation for advancing robust MAC and promotes the development of multimedia data mining. Our code is released in https://github.com/RH-Lin/MissMAC-Bench.
Decoding Ambiguous Emotions with Test-Time Scaling in Audio-Language Models
University of Melbourne(墨尔本大学);University of Auckland(奥克兰大学);South China Normal University(华南师范大学);MIT(麻省理工学院);University of Cambridge(剑桥大学);RMIT(皇家墨尔本理工大学)
Modeling emotion in complex stories: the Stanford Emotional Narratives Dataset
National University of Singapore(新加坡国立大学);A*STAR Artificial Intelligence Initiative, Agency for Science, Technology and Research(新加坡科技研究局人工智能计划);Stanford University(斯坦福大学);Massachusetts Institute of Technology(麻省理工学院);University of Oregon(俄勒冈大学)
Multimodal Large Language Models for End-to-End Affective Computing: Benchmarking and Boosting with Generative Knowledge Prompting
School of Computer Science, South China Normal University(华南师范大学计算机科学学院);School of Information Technology in Education, South China Normal University(华南师范大学教育信息技术学院)
StanceFlip: A Comprehensive Multi-Dimensional Benchmark for Multimodal Conversational Stance Flipping Forecasting
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件学院);Harbin Institute of Technology Shenzhen(哈尔滨工业大学(深圳));City University of Macau(澳门城市大学)
Cognition-Oriented Emotion Tracing from Causes to Consequences in Real-World Social Scenes
National University of Singapore(新加坡国立大学);School of Computing, National University of Singapore(新加坡国立大学计算机学院);University of Oxford(牛津大学);Institute of Computing and Intelligence, Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳)计算与智能研究院)
Multimodal Functional Maximum Correlation for Emotion Recognition
Key Laboratory of Child Development and Learning Science (Ministry of Education), School of Biological Sciences and Medical Engineering, Southeast University(儿童发展与学习科学重点实验室(教育部)、生物科学与医学工程学院、东南大学);Department of Artificial Intelligence, Westlake University(人工智能学院、西湖大学);Department of Artificial Intelligence, Vrije Universiteit Amsterdam(人工智能学院、阿姆斯特丹自由大学)
Edu-MMBias: A Three-Tier Multimodal Benchmark for Auditing Social Bias in Vision-Language Models under Educational Contexts
School of Computer Science and Technology, East China Normal University(东华师范大学计算机科学与技术学院);Shanghai Institute of Artificial Intelligence for Education, East China Normal University(东华师范大学教育人工智能研究所);Faculty of Education, East China Normal University(东华师范大学教育学院)
Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding
Shenzhen Technology University(深圳技术大学);Shenzhen University of Information Technology(深圳信息科技学院);University of Washington(华盛顿大学);Foundation Model Team, Meituan(美团基础模型团队);National University of Singapore(新加坡国立大学);Tongji University(同济大学)
MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
The Chinese University of Hong Kong(香港中文大学);Tongyi Lab(通义实验室);SZTU(深圳技术大学);Peking University(北京大学);Tongji University(同济大学);CASIA(中国科学院自动化所);Tencent(腾讯);UCLA(加州大学洛杉矶分校);Westlake University(西湖大学);NTU(国立大学)