针对误导性多模态内容的零样本警告生成
Zero-Shot Warning Generation for Misinformative Multimodal Content
AI总结:
提出基于跨模态一致性检查的多模态错误信息检测模型与轻量级模型,并引入零样本学习任务生成情境化警告以实现自动辟谣。
摘要:
错误信息的广泛存在引发了重大的社会担忧。脱离语境的错误信息将真实图像与虚假文本配对,尤其具有欺骗性且极易误导受众。大多数现有检测方法主要评估图文一致性,但往往缺乏充分的解释,而这对于有效辟谣至关重要。我们提出了一个通过跨模态一致性检查检测多模态错误信息的模型,仅需极少的训练时间。此外,我们提出了一个轻量级模型,仅使用三分之一的参数便实现了具有竞争力的性能。我们还引入了一个双用途零样本学习任务,用于生成情境化警告,从而实现自动辟谣并增强用户理解。对生成警告的定性评估和人类评估突显了我们方法的潜力与局限性。
英文摘要:
The widespread prevalence of misinformation poses significant societal concerns. Out-of-context misinformation, where authentic images are paired with false text, is particularly deceptive and easily misleads audiences. Most existing detection methods primarily evaluate image-text consistency but often lack sufficient explanations, which are essential for effectively debunking misinformation. We present a model that detects multimodal misinformation through cross-modality consistency checks, requiring minimal training time. Additionally, we propose a lightweight model that achieves competitive performance using only one-third of the parameters. We also introduce a dual-purpose zero-shot learning task for generating contextualized warnings, enabling automated debunking and enhancing user comprehension. Qualitative and human evaluations of the generated warnings highlight both the potential and limitations of our approach.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。