Meme-ingful分析：通过多模态解释增强对模因中网络欺凌的理解
Meme-ingful Analysis: Enhanced Understanding of Cyberbullying in Memes Through Multimodal Explanations
AI总结：
本文提出 MultiBully-Ex 代码混合网络欺凌模因多模态解释基准，并采用基于 CLIP 投影的共享-私有多任务方法，提升文本理由生成与视觉证据识别性能。
摘要：
互联网模因在传播政治、心理和社会文化观念方面已产生重大影响。虽然模因通常具有幽默性，但利用模因进行恶意挑衅和网络欺凌的现象正在增多。尽管人们已经开发了多种有效的基于深度学习的模型来检测攻击性多模态模因，但在可解释性方面开展的工作却很少。诸如《通用数据保护条例》中“解释权”之类的近期法律，推动了开发可解释模型的研究，而不再仅仅关注性能。受此启发，我们推出了 MultiBully-Ex，这是首个用于代码混合网络欺凌模因多模态解释的基准数据集。在该数据集中，视觉和文本模态都会被突出显示，用以解释给定模因为何属于网络欺凌。我们提出了一种基于对比语言-图像预训练（CLIP）投影的多模态共享-私有多任务方法，用于对模因进行视觉和文本解释。实验结果表明，利用多模态解释进行训练，提升了生成文本理由的性能，并能更准确地识别支持某一决策的视觉证据，带来了可靠的性能改进。
英文摘要：
Internet memes have gained significant influence in communicating political, psychological, and sociocultural ideas. While memes are often humorous, there has been a rise in the use of memes for trolling and cyberbullying. Although a wide variety of effective deep learning-based models have been developed for detecting offensive multimodal memes, only a few works have been done on explainability aspect. Recent laws like "right to explanations" of General Data Protection Regulation, have spurred research in developing interpretable models rather than only focusing on performance. Motivated by this, we introduce {\em MultiBully-Ex}, the first benchmark dataset for multimodal explanation from code-mixed cyberbullying memes. Here, both visual and textual modalities are highlighted to explain why a given meme is cyberbullying. A Contrastive Language-Image Pretraining (CLIP) projection-based multimodal shared-private multitask approach has been proposed for visual and textual explanation of a meme. Experimental results demonstrate that training with multimodal explanations improves performance in generating textual justifications and more accurately identifying the visual evidence supporting a decision with reliable performance improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
