SZTU-CMU at MER2024：利用卷积注意力机制改进 Emotion-LLaMA 以实现多模态情感识别
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition
AI总结：
本文提出利用 Emotion-LLaMA 生成高质量标注并引入 Conv-Attention 混合框架，有效缓解噪声并增强多模态融合，在 MER2024 挑战赛的 MER-NOISE 和 MER-OV 赛道中均取得最优成绩。
摘要：
本文介绍了我们在 MER2024 挑战赛多模态情感识别 MER-NOISE 和 MER-OV 赛道的获胜方法。我们的系统利用 Emotion-LLaMA 的高级情感理解能力，为无标签样本生成高质量标注，从而解决标注数据有限的挑战。为了增强多模态融合同时减轻特定模态噪声，我们引入了 Conv-Attention，这是一个轻量且高效的混合框架。大量实验验证了我们方法的有效性。在 MER-NOISE 赛道中，我们的系统实现了 85.30% 的最先进加权平均 F-score，分别比第二名和第三名团队高出 1.47% 和 1.65%。对于 MER-OV 赛道，我们利用 Emotion-LLaMA 进行开放词汇标注，与 GPT-4V 相比，平均准确率和召回率提升了 8.52%，在所有参与的大型多模态模型中获得最高分。Emotion-LLaMA 的代码和模型可在 https://github.com/ZebangCheng/Emotion-LLaMA 获取。
英文摘要：
This paper presents our winning approach for the MER-NOISE and MER-OV tracks of the MER2024 Challenge on multimodal emotion recognition. Our system leverages the advanced emotional understanding capabilities of Emotion-LLaMA to generate high-quality annotations for unlabeled samples, addressing the challenge of limited labeled data. To enhance multimodal fusion while mitigating modality-specific noise, we introduce Conv-Attention, a lightweight and efficient hybrid framework. Extensive experimentation vali-dates the effectiveness of our approach. In the MER-NOISE track, our system achieves a state-of-the-art weighted average F-score of 85.30%, surpassing the second and third-place teams by 1.47% and 1.65%, respectively. For the MER-OV track, our utilization of Emotion-LLaMA for open-vocabulary annotation yields an 8.52% improvement in average accuracy and recall compared to GPT-4V, securing the highest score among all participating large multimodal models. The code and model for Emotion-LLaMA are available at https://github.com/ZebangCheng/Emotion-LLaMA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
