M3Hop-CoT：基于多模态多跳思维链的厌女模因识别
M3Hop-CoT: Misogynous Meme Identification with Multimodal Multi-hop Chain-of-Thought
AI总结：
提出M3Hop-CoT框架，通过结合CLIP分类器和多模态CoT模块，利用三步提示原则整合情感、目标意识和语境知识，有效识别厌女模因。
摘要：
近年来，社交媒体平台上针对女性的仇恨现象显著增加，尤其是通过使用厌女模因。这些模因通常以微妙且隐蔽的线索针对女性，使得自动系统难以检测。近期，大型语言模型（LLMs）在使用思维链（CoT）提示生成中间推理链作为理据以促进多模态任务方面展现出良好效果，但往往忽略了文化多样性以及隐藏在视觉模态中的情感和语境知识等关键方面。为弥补这一空白，我们提出了一种用于厌女模因识别的多模态多跳CoT（M3Hop-CoT）框架，结合了基于CLIP的分类器和具有实体-对象-关系整合的多模态CoT模块。M3Hop-CoT采用三步多模态提示原则，以诱导情感、目标意识和语境知识用于模因分析。我们的实证评估（包括定性和定量分析）验证了M3Hop-CoT框架在SemEval-2022 Task 5（MAMI任务）数据集上的有效性，突出了其在宏观F1分数上的强劲表现。此外，我们通过在多个基准模因数据集上评估模型的可泛化性，为该方法在不同数据集上的有效性提供了全面见解。
英文摘要：
In recent years, there has been a significant rise in the phenomenon of hate against women on social media platforms, particularly through the use of misogynous memes. These memes often target women with subtle and obscure cues, making their detection a challenging task for automated systems. Recently, Large Language Models (LLMs) have shown promising results in reasoning using Chain-of-Thought (CoT) prompting to generate the intermediate reasoning chains as the rationale to facilitate multimodal tasks, but often neglect cultural diversity and key aspects like emotion and contextual knowledge hidden in the visual modalities. To address this gap, we introduce a Multimodal Multi-hop CoT (M3Hop-CoT) framework for Misogynous meme identification, combining a CLIP-based classifier and a multimodal CoT module with entity-object-relationship integration. M3Hop-CoT employs a three-step multimodal prompting principle to induce emotions, target awareness, and contextual knowledge for meme analysis. Our empirical evaluation, including both qualitative and quantitative analysis, validates the efficacy of the M3Hop-CoT framework on the SemEval-2022 Task 5 (MAMI task) dataset, highlighting its strong performance in the macro-F1 score. Furthermore, we evaluate the model's generalizability by evaluating it on various benchmark meme datasets, offering a thorough insight into the effectiveness of our approach across different datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
