Memes have gained popularity as a means to share visual ideas through the Internet and social media by mixing text, images and videos, often for humorous purposes. Research enabling automated analysis of memes has gained attention in recent years, including among others the task of classifying the emotion expressed in memes. In this paper, we propose a novel model, cluster-based deep ensemble learning (CDEL), for emotion classification in memes. CDEL is a hybrid model that leverages the benefits of a deep learning model in combination with a clustering algorithm, which enhances the model with additional information after clustering memes with similar facial features. We evaluate the performance of CDEL on a benchmark dataset for emotion classification, proving its effectiveness by outperforming a wide range of baseline models and achieving state-of-the-art performance. Further evaluation through ablated models demonstrates the effectiveness of the different components of CDEL.
MER-Bench: Benchmarking Multimodal Emotional Reappraisal for Image-Text Memes
School of Artificial Intelligence, Anhui University(安徽大学人工智能学院);Institute of Artificial Intelligence, Hefei Comprehensive National Science Center(合肥综合性国家科学中心人工智能研究院);School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机科学与信息工程学院);College of Information Technology, United Arab Emirates University(阿拉伯联合酋长国大学信息学院);Institute of Advanced Technology, University of Science and Technology of China(中国科学技术大学先进技术研究院)
Detecting Hate Speech in Memes Using Multimodal Deep Learning Approaches: Prize-winning solution to Hateful Memes Challenge
Bielefeld University(比勒费尔德大学);University of Bonn(波恩大学)
互联网上的模因通常是无害的,有时很有趣。然而,通过使用某些类型的图像,文本或两者的组合,看似无害的模因变成了一种多模态的仇恨言论-仇恨模因。Hateful Memes Challenge是首个专注于检测多模态模因中的仇恨言论的竞赛,它提出了一个包含10,000多个多模态内容新示例的新数据集。我们利用VisualBERT --这意味着是视觉和语言的BERT--它是在图像和字幕上进行多模态训练的,并应用了包围学习。我们的方法在挑战测试集上达到了0.811 AUROC,准确率为0.765,并在仇恨模因挑战的3,173名参与者中排名第三。
Dank or Not? -- Analyzing and Predicting the Popularity of Memes on Reddit
Aquincum Institute of Technology(阿昆库姆理工学院);Colorado College(科罗拉多学院);University of Kansas(堪萨斯大学);Haverford College(哈弗福德学院);Tufts University(塔夫茨大学);Dmlab Ltd.(Dmlab有限公司);MTA-BME Stochastics Research Group(MTA-BME随机研究组);Budapest University of Technology and Economics(布达佩斯技术与经济大学)