Internet memes are a powerful form of online communication, yet their nature and reliance on commonsense knowledge make toxicity detection challenging. Identifying key features for meme interpretation and understanding, is a crucial task. Previous work has been focused on some elements contributing to the meaning, such as the Textual dimension via OCR, the Visual dimension via object recognition, upper layers of meaning like the Emotional dimension, Toxicity detection via proxy variables, such as hate speech detection, and sentiment analysis. Nevertheless, there is still a lack of an overall architecture able to formally identify elements contributing to the meaning of a meme, and be used in the sense-making process. In this work, we present a semantic framework and a corresponding benchmark for automatic knowledge extraction from memes. First, we identify the necessary dimensions to understand and interpret a meme: Textual material, Visual material, Scene, Background Knowledge, Emotion, Semiotic Projection, Analogical Mapping, Overall Intent, Target Community, and Toxicity Assessment. Second, the framework guides a semi-automatic process of generating a benchmark with commonsense question-answer pairs about meme toxicity assessment and its underlying reason. The resulting benchmark M-QUEST consists of 609 question-answer pairs for 307 memes. Thirdly, we evaluate eight open-source large language models on their ability to correctly solve M-QUEST. Our results show that current models' commonsense reasoning capabilities for toxic meme interpretation vary depending on the dimension and architecture. Models with instruction tuning and reasoning capabilities significantly outperform the others, though pragmatic inference questions remain challenging. We release code, benchmark, and prompts to support future research intersecting multimodal content safety and commonsense reasoning.
Detecting and Understanding Hateful Contents in Memes Through Captioning and Visual Question-Answering
The University of Sydney, School of Computer Science（悉尼大学计算机科学学院）；University of Technology Sydney, School of Computer Science（新南威尔士大学技术学院）；University of Technology Sydney, TD School（新南威尔士大学TD学院）；Australian Catholic University, Peter Faber Business School（澳大利亚天主教大学彼得·法伯商学院）
KID: Knowledge-Injected Dual-Head Learning for Knowledge-Grounded Harmful Meme Detection
School of Economics and Management, Beijing University of Posts and Telecommunications（经济管理学院，北京邮电大学）；College of Computing, Beijing Information Science and Technology University（计算机学院，北京信息科技大学）
MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencing
Indraprastha Institute of Information Technology Delhi（因德拉普拉斯塔信息技术学院德里校区）；Indian Institute of Technology Delhi（印度理工学院德里校区）；Wipro R&D (Lab45)（威普罗研发中心（45实验室））；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
提出 MemeMQA 多模态问答框架、MemeMQACorpus 数据集与两阶段 ARSENAL 模型，用于兼具答案和理由的模因危害理解，准确率较最佳基线提升约18%。
KFinEval-Pilot: A Comprehensive Benchmark Suite for Korean Financial Language Understanding
Korea Financial Telecommunications and Clearings Institute（韩国金融电信与清算院）；Teddysum Inc.（泰迪萨姆公司）；SELECTSTAR Inc.（SELECTSTAR公司）；Konyang University（建阳大学）；Seoultech（首尔科技大学）；KAIST（韩国科学技术院）
MDIT-Bench: Evaluating the Dual-Implicit Toxicity in Large Multimodal Models
Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies（广东省新型安全智能技术重点实验室）；University of Barcelona（巴塞罗那大学）
AMAQA: A Metadata-based QA Dataset for RAG Systems
Institute for Informatics and Telematics, National Research Council（信息与电信研究院，国家研究院）；Department of Information Engineering, University of Pisa（信息工程系，比萨大学）；Department of Computer Science, University of Pisa（计算机科学系，比萨大学）
