arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.19585cs.MMcs.AI
收藏

三子空间解耦用于多模态情感分析

Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis

Fudan University(复旦大学);Peking University(北京大学);University of Macau(澳门大学)

Chunlei Meng, Jiabin Luo, Zhenglin Yan, Zhenyu Yu, Rong Fu, Zhongxue Gan, Chun Ouyang

AI总结:

本文提出三子空间解耦框架,通过分解多模态特征为公共、子模态共享和私人子空间,提升多模态情感分析的性能和鲁棒性。

摘要:

多模态情感分析(MSA)整合语言、视觉和听觉模态来推断人类情感。现有大多数方法要么专注于全局共享表示,要么专注于模态特定特征,而忽略了仅由某些模态对共享的信号。这限制了多模态表示的表达能力和判别能力。为了解决这一限制,我们提出了一个三子空间解耦(TSD)框架,该框架将特征显式分解为三个互补的子空间:一个捕捉全局一致性的公共子空间,子模态共享子空间建模成对的跨模态协同效应,以及私人子空间保留模态特定的提示。为了保持这些子空间的纯净和独立,我们引入了一个解耦监督器以及结构化正则化损失。我们进一步设计了一个子空间感知交叉注意(SACA)融合模块,该模块能够自适应地建模和整合来自三个子空间的信息,以获得更丰富和更稳健的表示。在CMU-MOSI和CMU-MOSEI上的实验表明,TSD在所有关键指标上均达到最佳性能,达到0.691 MAE在CMU-MOSI和54.9% ACC-7在CMU-MOSEI,并且也能很好地迁移到多模态意图识别任务中。消融研究证实,三子空间解耦和SACA联合增强了多粒度跨模态情感提示的建模。

英文摘要:

Multimodal Sentiment Analysis (MSA) integrates language, visual, and acoustic modalities to infer human sentiment. Most existing methods either focus on globally shared representations or modality-specific features, while overlooking signals that are shared only by certain modality pairs. This limits the expressiveness and discriminative power of multimodal representations. To address this limitation, we propose a Tri-Subspace Disentanglement (TSD) framework that explicitly factorizes features into three complementary subspaces: a common subspace capturing global consistency, submodally-shared subspaces modeling pairwise cross-modal synergies, and private subspaces preserving modality-specific cues. To keep these subspaces pure and independent, we introduce a decoupling supervisor together with structured regularization losses. We further design a Subspace-Aware Cross-Attention (SACA) fusion module that adaptively models and integrates information from the three subspaces to obtain richer and more robust representations. Experiments on CMU-MOSI and CMU-MOSEI demonstrate that TSD achieves state-of-the-art performance across all key metrics, reaching 0.691 MAE on CMU-MOSI and 54.9% ACC-7 on CMU-MOSEI, and also transfers well to multimodal intent recognition tasks. Ablation studies confirm that tri-subspace disentanglement and SACA jointly enhance the modeling of multi-granular cross-modal sentiment cues.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑