利用大型视觉语言模型检测和纠正多模态模因中的仇恨言论
Detecting and Correcting Hate Speech in Multimodal Memes with Large Visual Language Model
AI总结：
本研究利用零样本提示的视觉语言模型（如LLaVA）检测和纠正多模态模因中的仇恨言论，通过实证实验验证了其有效性并分析了优缺点。
摘要：
近年来，大型语言模型（LLMs）在自然语言处理领域备受瞩目。进一步地，将LLMs与视觉相结合，使用户能够探索多模态中更多的涌现能力。视觉语言模型（VLMs），如LLaVA、Flamingo或GPT-4，已在各种视觉-语言任务中展现出令人印象深刻的表现。因此，大型模型存在大量可能用于社交媒体平台的潜在应用。尽管如此，关于使用VLMs检测或纠正仇恨模因的相关工作仍然缺乏。在本研究中，我们通过零样本提示（zero-shot prompting）研究了VLMs在仇恨模因检测和仇恨模因纠正任务上的能力。通过我们的实证实验，我们展示了预训练LLaVA模型的有效性，并讨论了其在这些任务中的优点和缺点。
英文摘要：
Recently, large language models (LLMs) have taken the spotlight in natural language processing. Further, integrating LLMs with vision enables the users to explore more emergent abilities in multimodality. Visual language models (VLMs), such as LLaVA, Flamingo, or GPT-4, have demonstrated impressive performance on various visio-linguistic tasks. Consequently, there are enormous applications of large models that could be potentially used on social media platforms. Despite that, there is a lack of related work on detecting or correcting hateful memes with VLMs. In this work, we study the ability of VLMs on hateful meme detection and hateful meme correction tasks with zero-shot prompting. From our empirical experiments, we show the effectiveness of the pretrained LLaVA model and discuss its strengths and weaknesses in these tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
