AI 中文总结
研究如何检测和解释表情包中有害幽默，提出MAR-12框架，利用视觉语言模型，从十二个角度解读表情包，经注意力机制和原型分类器预测，在多数据集上准确率超现有方法，且能给出有说服力的解释。
AI 中文摘要
互联网表情包将视觉线索、文本内容和文化背景交织在一起，在幽默、讽刺和有害意图共存的场景中，其解读极具挑战性。现有多模态分类器要么忽略这些相互依存关系，要么提供的可解释性有限。本文介绍了MAR-12框架，它利用视觉语言模型在幽默和仇恨元素可能共存的环境中进行表情包检测和理解。该框架先从幽默和仇恨理论得出的十二个结构化角度解读每个表情包，再应用角色感知软门控注意力机制学习各角度的贡献程度，最后用基于原型的分类器进行最终预测。通过特定角度推理和学习到的注意力权重合成解释，确保解释透明且基于上下文。在PrideMM和Memotion数据集上评估，MAR-12在幽默检测中准确率高达80.3%，仇恨检测中为75.9%，优于现有方法。此外，人工和基于GPT-4的评估都证实MAR-12能给出连贯且有说服力的解释，尤其是对于幽默和有害线索同时出现的表情包。
英文摘要
Internet memes intertwine visual cues, textual content, and cultural context, making them particularly challenging to interpret in scenarios where humor, sarcasm, and harmful intent coexist. These complexities highlight the need for explainable meme understanding systems that can provide reliable and structured reasoning to support both accurate classification and human interpretability. However, existing multimodal classifiers either overlook these interdependencies or provide only limited interpretability. In this paper, we introduce MAR-12, a novel framework that leverages Vision Language Models (VLMs) for meme detection and understanding in settings where humorous and hateful elements may coexist. The framework first interprets each meme through twelve structured perspectives derived from humor and hate theories. It then applies a role-aware soft-gated attention mechanism to learn how much each perspective should contribute, followed by a prototype-based classifier for the final prediction. Finally, explanations are synthesized using both perspective-specific reasoning and learned attention weights, ensuring transparent and context-grounded justifications. We evaluate MAR-12 on the PrideMM and Memotion datasets, where it achieves up to 80.3% accuracy for humor detection and 75.9% accuracy for hate detection, outperforming state-of-the-art approaches. Furthermore, both human and GPT-4-based evaluations confirm that MAR-12 produces coherent and persuasive explanations, particularly for memes in which humorous and harmful cues co-occur.
CommentsAccepted for Publication at AAAI-ICWSM 2027
