MIAR: 多模态情感的模态交互与对齐表示融合
MIAR: Modality Interaction and Alignment Representation Fuison for Multimodal Emotion
AI总结：
MIAR通过模态交互与对齐表示融合，提升多模态情感识别的性能。
摘要：
多模态情感识别（MER）旨在通过语言、视觉和音频三种模式感知人类情感。先前的方法主要集中在模态融合上，但未能充分解决不同模态之间的显著分布差异，也未考虑其对任务的贡献差异。它们还缺乏在多样文本模型特征上进行稳健泛化的能力，从而限制了在多模态场景中的性能。因此，我们提出了一种新的方法，称为模态交互与对齐表示（MIAR）。该网络通过特征交互整合不同模态的上下文特征，生成特征标记以代表该模态从其他模态中提取信息的全局表示。这些四个标记代表每个模态从其他模态中提取信息的全局表示。MIAR通过对比学习和归一化策略对齐不同模态。我们在两个基准数据集上进行了实验：CMU-MOSI和CMU-MOSEI数据集，实验结果表明MIAR优于最先进的MER方法。
英文摘要：
Multimodal Emotion Recognition (MER) aims to perceive human emotions through three modes: language, vision, and audio. Previous methods primarily focused on modal fusion without adequately addressing significant distributional differences among modalities or considering their varying contributions to the task. They also lacked robust generalization capabilities across diverse textual model features, thus limiting performance in multimodal scenarios. Therefore, we propose a novel approach called Modality Interaction and Alignment Representation (MIAR). This network integrates contextual features across different modalities using a feature interaction to generate feature tokens to represent global representations of this modality extracting information from other modalities. These four tokens represent global representations of how each modality extracts information from others. MIAR aligns different modalities using contrastive learning and normalization strategies. We conduct experiments on two benchmarks: CMU-MOSI and CMU-MOSEI datasets, experimental results demonstrate the MIAR outperforms state-of-the-art MER methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
