发表机构
MFSDSAI; Indian Institute of Technology Guwahati（MFSDSAI; 印度理工学院古瓦哈提分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出MemeCF基准和MemePIVOT架构，通过识别关键线索提升多模态模因分类的鲁棒性，实验显示优于现有基线。
AI 中文摘要
模因的有害、仇恨或讽刺含义往往源于微小但决定性的视觉、文本或跨模态线索。现有依赖全局图像-文本表示的多模态分类器可能会遗漏此类证据。我们引入了MemeCF，一个包含9,895个模因的线索聚焦基准，涵盖危害、仇恨和讽刺三类，并带有标注以识别关键证据的模态和理由。我们还提出了MemePIVOT，一种用于模因分类的局部-全局架构。MemePIVOT使用冻结的CLIP特征，通过不平衡最优传输将单词与图像块对齐，同时允许不相关的证据保持不匹配，并采用证据融合头在不确定性下将局部定位与全局模因上下文结合。在HarMeme、PrideMM和MemeCF上的实验显示，相较于强文本-only、图像-only、多模态和视觉-语言基线，取得了持续改进。跨数据集和消融结果进一步表明，显式关键证据建模提高了鲁棒性，并在全局多模态表示之外做出了有意义的贡献。我们的代码和数据集公开于https://github.com/AkshitSharma1/MemePIVOT。
英文摘要
Memes often derive their harmful, hateful, or sarcastic meaning from small but decisive visual, textual, or cross-modal cues. Existing multimodal classifiers can miss such evidence when relying mainly on global image-text representations. We introduce MemeCF, a cue-focused benchmark of 9,895 memes across harm, hate, and sarcasm, with annotations identifying the modality and rationale of the pivotal evidence. We also propose MemePIVOT, a local-global architecture for meme classification. MemePIVOT uses frozen CLIP features, unbalanced optimal transport to align words with image patches while allowing irrelevant evidence to remain unmatched, and an evidential fusion head to combine local grounding with global meme context under uncertainty. Experiments on HarMeme, PrideMM, and MemeCF show consistent gains over strong text-only, image-only, multimodal, and vision-language baselines. Cross-dataset and ablation results further show that explicit pivotal-evidence modeling improves robustness and contributes meaningfully beyond global multimodal representations. Our code and dataset are publicly available at https://github.com/AkshitSharma1/MemePIVOT
CommentsAccepted to EMNLP 2026 Findings
