有害模因中的实体刻画：谁是英雄、反派与受害者？
Characterizing the Entities in Harmful Memes: Who is the Hero, the Villain, the Victim?
AI总结：
本文针对有害模因中实体角色识别任务，提出多模态框架VECTOR，通过融合实体上下文信息，在HVVMemes数据集上比最佳基线和最佳竞赛提交分别提升4%和1%。
摘要：
模因（memes）以易于消费的方式结合视觉与文本信息，能够在社交媒体上影响人们的观点。由于模因会迅速病毒式传播，推断其意图及潜在关联的有害性对于及时采取必要措施至关重要。模因理解中一个常见问题在于检测所引用的实体并刻画每个实体的角色。本文旨在理解模因是否美化、贬损或伤害其所指涉的每个实体。为此，我们处理有害模因中实体角色识别的任务，即检测模因中谁是“英雄”、“反派”和“受害者”（如果存在）。我们利用HVVMemes——一个关于美国政治和Covid-19模因的数据集，该数据集最近作为CONSTRAINT@ACL-2022共享任务的一部分发布。它包含模因、所引用的实体及其关联角色：英雄、反派、受害者和其他。我们进一步设计了VECTOR（Visual-semantic role dEteCToR），一个用于该任务的鲁棒多模态框架，它在多模态表示中整合了基于实体的上下文信息，并将其与若干标准单模态（仅文本或仅图像）或多模态（图像+文本）模型进行比较。我们的实验结果表明，所提出的模型比最佳基线提高了4%，比共享任务中最佳竞争性独立提交提高了1%。除了展示带有比较分析的广泛实验设置外，我们最后还强调了在处理模因内语义角色标注这一复杂任务时所遇到的挑战。
英文摘要：
Memes can sway people's opinions over social media as they combine visual and textual information in an easy-to-consume manner. Since memes instantly turn viral, it becomes crucial to infer their intent and potentially associated harmfulness to take timely measures as needed. A common problem associated with meme comprehension lies in detecting the entities referenced and characterizing the role of each of these entities. Here, we aim to understand whether the meme glorifies, vilifies, or victimizes each entity it refers to. To this end, we address the task of role identification of entities in harmful memes, i.e., detecting who is the 'hero', the 'villain', and the 'victim' in the meme, if any. We utilize HVVMemes - a memes dataset on US Politics and Covid-19 memes, released recently as part of the CONSTRAINT@ACL-2022 shared-task. It contains memes, entities referenced, and their associated roles: hero, villain, victim, and other. We further design VECTOR (Visual-semantic role dEteCToR), a robust multi-modal framework for the task, which integrates entity-based contextual information in the multi-modal representation and compare it to several standard unimodal (text-only or image-only) or multi-modal (image+text) models. Our experimental results show that our proposed model achieves an improvement of 4% over the best baseline and 1% over the best competing stand-alone submission from the shared-task. Besides divulging an extensive experimental setup with comparative analyses, we finally highlight the challenges encountered in addressing the complex task of semantic role labeling within memes.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
