Mixture of Disentangled Experts with Missing Modalities for Robust Multimodal Sentiment Analysis
School of Cyber Science and Technology, Beihang University(北京航空航天大学网络空间安全学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机学院);School of Information Science and Technology, Beijing University of Technology(北京工业大学信息学部)
Multimodal Sentiment Analysis (MSA) integrates multiple modalities to infer human sentiment, but real-world noise often leads to missing or corrupted data. However, existing feature-disentangled methods struggle to handle the internal variations of heterogeneous information under uncertain missingness, making it difficult to learn effective multimodal representations from degraded modalities. To address this issue, we propose DERL, a Disentangled Expert Representation Learning framework for robust MSA. Specifically, DERL employs hybrid experts to adaptively disentangle multimodal inputs into orthogonal private and shared representation spaces. A multi-level reconstruction strategy is further developed to provide collaborative supervision, enhancing both the expressiveness and robustness of the learned representations. Finally, the disentangled features act as modality experts with distinct roles to generate importance-aware fusion results. Extensive experiments on two MSA benchmarks demonstrate that DERL outperforms state-of-the-art methods under various missing-modality conditions. For instance, our method achieves improvements of 2.47% in Acc-2 and 2.25% in MAE on MOSI under intra-modal missingness.
TF-Mamba: Text-enhanced Fusion Mamba with Missing Modalities for Robust Multimodal Sentiment Analysis
School of Cyber Science and Technology, Beihang University(北京航空航天大学网络空间安全学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院)
School of Computer Science and Engineering, NTU(南洋理工大学计算机科学与工程学院);School of Natural Sciences, University of Stirling(斯特灵大学自然科学学院);Temasek Laboratories, NTU(南洋理工大学淡马锡实验室);National Institute of Technology, Warangal(瓦朗加尔国立理工学院)
TCAN: Text-oriented Cross Attention Network for Multimodal Sentiment Analysis
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);Tencent(腾讯);School of Information Science and Technology, Donghua University(东华大学信息科学与技术学院)
Divide and Conquer: Mixture-of-Bottleneck Experts in Informative Ordinal Space for Video-based Multimodal Sentiment Analysis
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院);School of Electronics and Information Technology, Sun Yat-sen University(中山大学电子信息工程学院);School of Cyberspace Security, Guangzhou University(广州大学网络空间安全学院);School of Computer Science, South China Normal University(华南师范大学计算机学院);VinUniversity(越南河内VinUniversity);Nanyang Technological University(南洋理工大学);School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电气与电子工程学院);Desay SV Automotive Co., Ltd(德赛西威汽车电子股份有限公司)
Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection
University of Macau(澳门大学);Zhejiang University(浙江大学);Tsinghua University(清华大学);Tongji University(同济大学);University of Southampton(南安普顿大学);University of Bologna(博洛尼亚大学);Minzu University of China(中央民族大学)
Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module
Tsinghua University(清华大学);Hebei University of Science and Technology(河北科技大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Beijing University of Posts and Telecommunications(北京邮电大学);China Agricultural University(中国农业大学)