Improving model robustness against potential modality noise, as an essential step for adapting multimodal models to real-world applications, has received increasing attention among researchers. For Multimodal Sentiment Analysis (MSA), there is also a debate on whether multimodal models are more effective against noisy features than unimodal ones. Stressing on intuitive illustration and in-depth analysis of these concerns, we present Robust-MSA, an interactive platform that visualizes the impact of modality noise as well as simple defence methods to help researchers know better about how their models perform with imperfect real-world data.
TF-Mamba: Text-enhanced Fusion Mamba with Missing Modalities for Robust Multimodal Sentiment Analysis
School of Cyber Science and Technology, Beihang University(北京航空航天大学网络空间安全学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机科学与工程学院)
Mixture of Disentangled Experts with Missing Modalities for Robust Multimodal Sentiment Analysis
School of Cyber Science and Technology, Beihang University(北京航空航天大学网络空间安全学院);School of Computer Science and Engineering, Beihang University(北京航空航天大学计算机学院);School of Information Science and Technology, Beijing University of Technology(北京工业大学信息学部)
TCAN: Text-oriented Cross Attention Network for Multimodal Sentiment Analysis
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences(多模态人工智能系统国家重点实验室,中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);Tencent(腾讯);School of Information Science and Technology, Donghua University(东华大学信息科学与技术学院)
Divide and Conquer: Mixture-of-Bottleneck Experts in Informative Ordinal Space for Video-based Multimodal Sentiment Analysis
College of Computer Science and Software Engineering, Shenzhen University(深圳大学计算机科学与软件工程学院);School of Electronics and Information Technology, Sun Yat-sen University(中山大学电子信息工程学院);School of Cyberspace Security, Guangzhou University(广州大学网络空间安全学院);School of Computer Science, South China Normal University(华南师范大学计算机学院);VinUniversity(越南河内VinUniversity);Nanyang Technological University(南洋理工大学);School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电气与电子工程学院);Desay SV Automotive Co., Ltd(德赛西威汽车电子股份有限公司)
Mitigating Shared-Private Branch Imbalance via Dual-Branch Rebalancing for Multimodal Sentiment Analysis
the College of Intelligent Robotics and Advanced Manufacturing, Fudan University(复旦大学智能机器人与先进制造学院);Peking University(北京大学);University of Southern California(南加州大学);China University of Petroleum-Beijing at Karamay(中国石油大学(北京)克拉玛依校区)
Improving the Modality Representation with Multi-View Contrastive Learning for Multimodal Sentiment Analysis
University of Chinese Academy of Sciences(中国科学院大学);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);Henan University(河南大学)
Towards Robust Multimodal Sentiment Analysis with Incomplete Data
School of Data Science, The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳)数据科学学院);University College London(伦敦大学学院);School of Computer Science, Wuhan University(武汉大学计算机学院)
Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysis
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));China University of Geosciences, Wuhan(中国地质大学(武汉));Nanyang Technological University(南洋理工大学);Shenzhen Institute of Artificial Intelligence and Robotics for Society(深圳人工智能与机器人研究院)
M-SENA: An Integrated Platform for Multimodal Sentiment Analysis
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Hebei University of Science and Technology(河北科技大学)
Read Between the Stickers: Sentiment-Prior Reasoning with Learnable Verbalized Rules for Multimodal Chat Analysis
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件学院);College of Computer Science and Technology, Xi’an University of Electronic Science and Technology(西安电子科技大学计算机科学与技术学院);Nexus Lab, Peking University(北京大学Nexus实验室);Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术);School of education, Zhejiang University(浙江大学教育学院);Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);National Engineering Laboratory for Big Data Analysis and Applications, Peking University(北京大学大数据分析与应用技术国家工程实验室)
Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module
Tsinghua University(清华大学);Hebei University of Science and Technology(河北科技大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Beijing University of Posts and Telecommunications(北京邮电大学);China Agricultural University(中国农业大学)
Emotion Collider: Dual Hyperbolic Mirror Manifolds for Sentiment Recovery via Anti Emotion Reflection
University of Macau(澳门大学);Zhejiang University(浙江大学);Tsinghua University(清华大学);Tongji University(同济大学);University of Southampton(南安普顿大学);University of Bologna(博洛尼亚大学);Minzu University of China(中央民族大学)