arXiv 2509.174462026-06-08cs.LGcs.AI

MVCL-DAF++: 通过原型感知对比对齐和由粗到细动态注意力融合增强多模态意图识别

MVCL-DAF++: Enhancing Multimodal Intent Recognition via Prototype-Aware Contrastive Alignment and Coarse-to-Fine Dynamic Attention Fusion

University of Shanghai for Science and TechnologyChina（上海科学技术大学中国）；Shenzhen Institute of Advanced Technology, Chinese Academy of SciencesChina（深圳先进技术研究院，中国科学院中国）；University of Minnesota-Twin Cities, USA（明尼苏达大学双城分校，美国）；University of LeedsUK（利兹大学，英国）

Haofeng Huang, Yifei Han, Long Zhang, Bin Li, Yangfan He, Yaxin Xue

备注：: Accepted by Interspeech 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出MVCL-DAF++，通过原型感知对比对齐和由粗到细注意力融合，在MIntRec和MIntRec2.0上提升多模态意图识别，尤其改善稀有类识别。

摘要：

多模态意图识别（MMIR）在噪声或稀有类条件下存在语义基础薄弱和鲁棒性差的问题。我们提出MVCL-DAF++，它通过两个关键模块扩展了MVCL-DAF：（1）原型感知对比对齐，将实例与类级原型对齐以增强语义一致性；（2）由粗到细注意力融合，将全局模态摘要与令牌级特征集成以实现层次化跨模态交互。在MIntRec和MIntRec2.0上，MVCL-DAF++取得了新的最佳结果，稀有类识别WF1分别提高了+1.05%和+4.18%。这些结果证明了原型引导学习和由粗到细融合对于鲁棒多模态理解的有效性。源代码可在https://github.com/chr1s623/MVCL-DAF-PlusPlus获取。

英文摘要：

Multimodal intent recognition (MMIR) suffers from weak semantic grounding and poor robustness under noisy or rare-class conditions. We propose MVCL-DAF++, which extends MVCL-DAF with two key modules: (1) Prototype-aware contrastive alignment, aligning instances to class-level prototypes to enhance semantic consistency; and (2) Coarse-to-fine attention fusion, integrating global modality summaries with token-level features for hierarchical cross-modal interaction. On MIntRec and MIntRec2.0, MVCL-DAF++ achieves new state-of-the-art results, improving rare-class recognition by +1.05\% and +4.18\% WF1, respectively. These results demonstrate the effectiveness of prototype-guided learning and coarse-to-fine fusion for robust multimodal understanding. The source code is available at https://github.com/chr1s623/MVCL-DAF-PlusPlus.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。