CorMulT：一种用于情感分析的半监督模态相关性感知多模态Transformer
CorMulT: A Semi-supervised Modality Correlation-aware Multimodal Transformer for Sentiment Analysis
AI总结：
针对弱相关多模态数据情感识别难题，提出两阶段半监督模型CorMulT，通过对比学习模态相关系数并融合模态表示，在CMU-MOSEI上显著超越现有方法。
摘要：
多模态情感分析是一个活跃的研究领域，它结合文本、图像和音频等多种数据模态来分析人类情绪，并惠及多种应用。现有的多模态情感分析方法可分为基于模态交互的方法、基于模态变换的方法和基于模态相似性的方法。然而，这些方法大多高度依赖模态之间的强相关性，无法充分挖掘和利用模态间的相关性来增强情感分析。因此，这些方法在识别弱相关性多模态数据的情感时通常表现不佳。为解决这一问题，我们提出了一种两阶段半监督模型，称为相关性感知多模态Transformer（CorMulT），该模型包含预训练阶段和预测阶段。在预训练阶段，设计了一个模态相关性对比学习模块，以高效学习不同模态之间的模态相关系数。在预测阶段，将学习到的相关系数与模态表示融合以进行情感预测。根据在流行的多模态数据集CMU-MOSEI上的实验，CorMulT明显超越了最先进的多模态情感分析方法。
英文摘要：
Multimodal sentiment analysis is an active research area that combines multiple data modalities, e.g., text, image and audio, to analyze human emotions and benefits a variety of applications. Existing multimodal sentiment analysis methods can be classified as modality interaction-based methods, modality transformation-based methods and modality similarity-based methods. However, most of these methods highly rely on the strong correlations between modalities, and cannot fully uncover and utilize the correlations between modalities to enhance sentiment analysis. Therefore, these methods usually achieve bad performance for identifying the sentiment of multimodal data with weak correlations. To address this issue, we proposed a two-stage semi-supervised model termed Correlation-aware Multimodal Transformer (CorMulT) which consists pre-training stage and prediction stage. At the pre-training stage, a modality correlation contrastive learning module is designed to efficiently learn modality correlation coefficients between different modalities. At the prediction stage, the learned correlation coefficients are fused with modality representations to make the sentiment prediction. According to the experiments on the popular multimodal dataset CMU-MOSEI, CorMulT obviously surpasses state-of-the-art multimodal sentiment analysis methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
