发表机构
School of Computer Science and Technology, Xinjiang University; Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing; Xinjiang Multimodal Intelligent Processing and Information Security Engineering Technology Research Center; Pengcheng Laboratory Xinjiang Network Node; Embodied Intelligence Joint Laboratory（新疆大学计算机科学与技术学院; 丝绸之路多语言认知计算国际联合研究实验室; 新疆多模态智能处理与信息安全工程技术研究中心; 鹏城实验室新疆网络节点; 具身智能联合实验室）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多模态情感分析中模态质量不一问题，提出MRUF方法，结合多粒度路由与不确定性感知校准，总结情感表示、进行路由并监督模态重要性估计，预测模态不确定性并细化模态门，实验显示相比基线有改进，验证了高不确定性模态获低融合权重。
AI 中文摘要
多模态情感分析依赖语言、视觉和声学线索，但话语级模态质量可能因遮挡、背景噪声、运动模糊或不完美转录而变化，导致传统融合过度信任不可靠模态。我们提出了MRUF，一种可靠性感知融合方法，它将多粒度路由与不确定性感知校准相结合。MRUF总结与情感相关的表示，执行子空间和模态级路由，并用留一法误差增加来监督模态路由以估计话语级模态重要性。它还预测模态不确定性并通过逆方差重新加权来细化模态门，同时模态不变对比对齐稳定共享表示空间。在CMU - MOSI和CMU - MOSEI上的实验表明，与强大基线相比有持续改进，机制分析验证了预测不确定性较高的模态获得较低的融合权重。
英文摘要
Multimodal sentiment analysis relies on language, visual, and acoustic cues, but utterance-level modality quality may vary due to occlusion, background noise, motion blur, or imperfect transcripts, causing conventional fusion to over-trust unreliable modalities. We propose MRUF, a reliability-aware fusion method that combines multi-granularity routing with uncertainty-aware calibration. MRUF summarizes sentiment-relevant representations, performs subspace- and modality-level routing, and supervises modality routing with leave-one-out error increases to estimate utterance-level modality importance. It further predicts modality-wise uncertainty and refines modality gates through inverse-variance reweighting, while modality-invariant contrastive alignment stabilizes the shared representation space. Experiments on CMU-MOSI and CMU-MOSEI under aligned and unaligned settings show consistent improvements over strong baselines, and mechanism analysis verifies that modalities with higher predicted uncertainty receive lower fusion weights.
CommentsMain paper (6 pages). Accepted for publication by IEEE International Conference on Systems and Man and and Cybernetics 2026 (IEEE SMC 2026)
