检测与理解模因中的仇恨内容：通过字幕生成与视觉问答
Detecting and Understanding Hateful Contents in Memes Through Captioning and Visual Question-Answering
AI总结：
提出一种整合OCR、字幕生成、子标签分类、RAG和VQA的多模态框架，用于检测模因中的仇恨内容，在Facebook数据集上超越现有模型。
摘要：
模因广泛用于幽默和文化评论，但越来越多地被利用来传播仇恨内容。由于其多模态特性，仇恨模因往往能逃避传统的仅文本或仅图像检测系统，尤其是当它们采用微妙或编码的引用时。为应对这些挑战，我们提出了一种多模态仇恨检测框架，该框架整合了关键组件：OCR用于提取嵌入文本，字幕生成用于中性描述视觉内容，子标签分类用于仇恨内容的细粒度分类，RAG用于上下文相关的检索，以及VQA用于对符号和上下文线索进行迭代分析。这使得框架能够揭示简单流程无法检测到的潜在信号。在Facebook仇恨模因数据集上的实验结果表明，所提出的框架在准确性和AUC-ROC方面均超过了单模态和传统多模态模型的性能。
英文摘要：
Memes are widely used for humor and cultural commentary, but they are increasingly exploited to spread hateful content. Due to their multimodal nature, hateful memes often evade traditional text-only or image-only detection systems, particularly when they employ subtle or coded references. To address these challenges, we propose a multimodal hate detection framework that integrates key components: OCR to extract embedded text, captioning to describe visual content neutrally, sub-label classification for granular categorization of hateful content, RAG for contextually relevant retrieval, and VQA for iterative analysis of symbolic and contextual cues. This enables the framework to uncover latent signals that simpler pipelines fail to detect. Experimental results on the Facebook Hateful Memes dataset reveal that the proposed framework exceeds the performance of unimodal and conventional multimodal models in both accuracy and AUC-ROC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
