检测跨模态不一致性以防御神经假新闻
Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News
摘要：
旨在误导或欺骗公众的大规模网上虚假信息传播是一个重大的社会问题。图像、视频和自然语言生成模型的快速发展只会加剧这种情况，并加剧我们对有效防御机制的需求。虽然已经提出了现有的方法来防御神经假新闻，但它们通常局限于非常有限的设置，其中文章只有文本和元数据，如标题和作者。在本文中，我们介绍了更现实和更具挑战性的任务，防御机器生成的新闻，也包括图像和字幕。为了识别对手可能利用的弱点，我们创建了一个由4种不同类型的生成文章组成的NeuralNews数据集，并基于该数据集进行了一系列人类用户研究实验。除了从我们的用户研究实验中收集到的有价值的见解之外，我们还提供了一种基于检测视觉语义不一致的相对有效的方法，这将作为有效的第一道防线，并为未来防御机器生成的虚假信息的工作提供有用的参考。
英文摘要：
Large-scale dissemination of disinformation online intended to mislead or deceive the general population is a major societal problem. Rapid progression in image, video, and natural language generative models has only exacerbated this situation and intensified our need for an effective defense mechanism. While existing approaches have been proposed to defend against neural fake news, they are generally constrained to the very limited setting where articles only have text and metadata such as the title and authors. In this paper, we introduce the more realistic and challenging task of defending against machine-generated news that also includes images and captions. To identify the possible weaknesses that adversaries can exploit, we create a NeuralNews dataset composed of 4 different types of generated articles as well as conduct a series of human user study experiments based on this dataset. In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies, which will serve as an effective first line of defense and a useful reference for future work in defending against machine-generated disinformation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
