面向多模态反讽检测的带反讽感知对比正则化的动态门控跨模态融合方法
Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization for Multimodal Sarcasm Detection
浏览论文内容
中文总结 AI 辅助
该研究针对多模态反讽检测任务，提出集成动态门控跨模态融合与SaCR正则化的MSD框架，经实验验证其性能优于多个强基线模型。
中文摘要 AI 辅助
多模态反讽检测旨在从多模态内容中识别反讽意图，字面含义与上下文线索之间的不一致往往是反讽的信号，该任务已受到越来越多的研究关注。然而，由于实例相关的模态贡献以及误导性的语义一致性，准确检测仍具挑战性，表面层面的对齐会掩盖潜在的矛盾意图。现有方法通常依赖固定的融合策略，并将反讽视为通用的跨模态不匹配，限制了其捕捉细微反讽线索和实例特定模态交互的能力。为应对这些挑战，我们提出了一种新颖的多模态反讽检测（MSD）框架，该框架集成了动态门控跨模态融合与反讽感知对比正则化（SaCR）。具体而言，双向门控交互模块执行跨模态特征过滤，并在实例层面自适应校准文本和视觉贡献；动态融合门进一步平衡模态重要性，以生成更鲁棒的多模态表示。此外，SaCR作为一种感知标签的对比正则化目标，鼓励非反讽样本的语义一致性，同时抑制反讽案例中的误导性一致性。所提框架采用多目标学习策略进行端到端训练，该策略联合优化多模态分类和辅助单模态监督。在MMSD和MMSD2.0上开展的大量实验表明，所提方法始终优于多个强基线模型。
英文摘要
Multimodal sarcasm detection aims to identify sarcastic intent from multimodal content, where inconsistencies between literal meaning and contextual cues often signal irony. This task has attracted increasing research attention. However, accurate detection remains challenging due to instance-dependent modality contributions and misleading semantic consistency, where surface-level alignment masks underlying contradictory intent. Existing methods often rely on fixed fusion strategies and treat sarcasm as generic cross-modal mismatch, limiting their ability to capture subtle sarcasm cues and instance-specific modality interactions. To address these challenges, we propose a novel MSD framework that integrates Dynamic Gated Cross-Modal Fusion with Sarcastic-aware Contrastive Regularization (SaCR). Specifically, a bidirectional gated interaction module performs cross-modal feature filtering and adaptively calibrates textual and visual contributions at the instance level. A dynamic fusion gate further balances modality importance to generate more robust multimodal representations. Furthermore, SaCR is introduced as a label-aware contrastive regularization objective that encourages semantic consistency for non-sarcastic samples while suppressing misleading consistency in sarcastic cases. The proposed framework is trained end-to-end with a multi-objective learning strategy that jointly optimizes multimodal classification and auxiliary unimodal supervision. Extensive experiments on MMSD and MMSD2.0 demonstrate that the proposed method consistently outperforms strong baselines.
发表机构
- Renmin University of China（中国人民大学）
- Anhui Polytechnic University（安徽工程大学）
机构由 AI 辅助整理，请以论文原文为准。
