多模态假新闻检测：MFND 数据集与浅层-深层多任务学习
Multimodal Fake News Detection: MFND Dataset and Shallow-Deep Multitask Learning
AI总结：
本文构建含11种篡改类型的MFND多模态假新闻数据集，并提出SDML浅层-深层多任务学习模型，通过对比对齐、跨模态融合和双分支预测实现高逼真假新闻的检测与定位。
摘要：
多模态新闻包含丰富的信息，且容易受到 deepfake 建模攻击的影响。为对抗最新的图像和文本生成方法，我们提出了一个新的多模态假新闻检测数据集（Multimodal Fake News Detection，MFND），其中包含 11 种篡改类型，旨在检测并定位高度逼真的假新闻。此外，我们提出了一种用于假新闻的浅层-深层多任务学习（Shallow-Deep Multitask Learning，SDML）模型，该模型充分利用单模态和互模态特征来挖掘新闻的内在语义。在浅层推理下，我们提出了基于动量蒸馏的轻惩罚对比学习，用于细粒度、统一的空间图像与文本语义对齐，并提出自适应跨模态融合模块以增强互模态特征。在深层推理下，我们设计了一个双分支框架来增强图像和文本单模态特征，并分别将其与互模态特征合并，通过专用的检测和定位投影得到四项预测。在主流数据集和我们提出的数据集上的实验均证明了该模型的优越性。代码和数据集已发布于 https://github.com/yunan-wang33/sdml。
英文摘要：
Multimodal news contains a wealth of information and is easily affected by deepfake modeling attacks. To combat the latest image and text generation methods, we present a new Multimodal Fake News Detection dataset (MFND) containing 11 manipulated types, designed to detect and localize highly authentic fake news. Furthermore, we propose a Shallow-Deep Multitask Learning (SDML) model for fake news, which fully uses unimodal and mutual modal features to mine the intrinsic semantics of news. Under shallow inference, we propose the momentum distillation-based light punishment contrastive learning for fine-grained uniform spatial image and text semantic alignment, and an adaptive cross-modal fusion module to enhance mutual modal features. Under deep inference, we design a two-branch framework to augment the image and text unimodal features, respectively merging with mutual modalities features, for four predictions via dedicated detection and localization projections. Experiments on both mainstream and our proposed datasets demonstrate the superiority of the model. Codes and dataset are released at https://github.com/yunan-wang33/sdml.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
