TopicDiff：一种面向多模态对话情绪检测的主题增强扩散方法
TopicDiff: A Topic-enriched Diffusion Approach for Multimodal Conversational Emotion Detection
AI总结：
针对现有多模态对话情绪检测研究忽略声学、视觉模态主题信息的问题，提出将扩散模型集成到神经主题模型的TopicDiff方法，有效捕获多模态主题信息，性能超越现有最优基准，且发现声学、视觉主题信息比语言的更具区分度和鲁棒性。
摘要：
多模态对话情绪（MCE）检测通常涵盖声学、视觉和语言模态，已在多媒体领域引发越来越多的研究兴趣。以往研究主要聚焦于学习对话中的上下文信息，仅有少数研究考虑了单一语言模态中的主题信息，且始终忽略声学和视觉主题信息。在此基础上，我们提出一种模型无关的主题增强扩散（TopicDiff）方法，用于在MCE任务中捕获多模态主题信息。具体而言，我们将扩散模型集成到神经主题模型中，以缓解神经主题模型在捕获主题信息时的多样性不足问题。详细评估表明，TopicDiff相比当前最优的MCE基准模型取得了显著提升，证明了多模态主题信息对MCE的重要性，以及TopicDiff捕获此类信息的有效性。此外，我们还发现了一个有趣的结论：与语言模态相比，声学和视觉中的主题信息更具区分度和鲁棒性。
英文摘要：
Multimodal Conversational Emotion (MCE) detection, generally spanning across the acoustic, vision and language modalities, has attracted increasing interest in the multimedia community. Previous studies predominantly focus on learning contextual information in conversations with only a few considering the topic information in single language modality, while always neglecting the acoustic and vision topic information. On this basis, we propose a model-agnostic Topic-enriched Diffusion (TopicDiff) approach for capturing multimodal topic information in MCE tasks. Particularly, we integrate the diffusion model into neural topic model to alleviate the diversity deficiency problem of neural topic model in capturing topic information. Detailed evaluations demonstrate the significant improvements of TopicDiff over the state-of-the-art MCE baselines, justifying the importance of multimodal topic information to MCE and the effectiveness of TopicDiff in capturing such information. Furthermore, we observe an interesting finding that the topic information in acoustic and vision is more discriminative and robust compared to the language.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
