细粒度多重监督网络用于多模态篡改检测与定位
Fine-grained Multiple Supervisory Network for Multi-modal Manipulation Detecting and Grounding
AI总结：
针对多模态媒体篡改检测与定位任务，提出细粒度多重监督网络，通过模态可靠性、单模态内部及跨模态三重监督，有效提升检测性能。
摘要：
检测与定位多模态媒体篡改（DGM$^4$）任务是虚假信息检测的一个分支。与传统的二分类不同，它包含诸如伪造内容定位和伪造方法分类等复杂子任务。考虑到现有方法往往因忽视不可靠单模态数据造成的错误干扰，以及未能建立全面的伪造监督来挖掘细粒度篡改痕迹而性能受限，本文提出了一种细粒度多重监督（FMS）网络，该网络融合了模态可靠性监督、单模态内部监督和跨模态监督，为DGM$^4$检测提供全面指导。对于模态可靠性监督，我们提出了多模态决策监督校正（MDSC）模块，利用单模态弱监督来校正多模态决策过程。对于单模态内部监督，我们提出了单模态伪造挖掘强化（UFMR）模块，从特征级和样本级两个角度放大单模态内真实与虚假信息之间的差异。对于跨模态监督，我们提出了多模态伪造对齐推理（MFAR）模块，利用软注意力交互从一致性和不一致性两个角度实现跨模态特征感知，并设计了交互约束以确保交互质量。大量实验表明，我们的FMS相比最先进的方法具有优越性能。
英文摘要：
The task of Detecting and Grounding Multi-Modal Media Manipulation (DGM$^4$) is a branch of misinformation detection. Unlike traditional binary classification, it includes complex subtasks such as forgery content localization and forgery method classification. Consider that existing methods are often limited in performance due to neglecting the erroneous interference caused by unreliable unimodal data and failing to establish comprehensive forgery supervision for mining fine-grained tampering traces. In this paper, we present a Fine-grained Multiple Supervisory (FMS) network, which incorporates modality reliability supervision, unimodal internal supervision and cross-modal supervision to provide comprehensive guidance for DGM$^4$ detection. For modality reliability supervision, we propose the Multimodal Decision Supervised Correction (MDSC) module. It leverages unimodal weak supervision to correct the multi-modal decision-making process. For unimodal internal supervision, we propose the Unimodal Forgery Mining Reinforcement (UFMR) module. It amplifies the disparity between real and fake information within unimodal modality from both feature-level and sample-level perspectives. For cross-modal supervision, we propose the Multimodal Forgery Alignment Reasoning (MFAR) module. It utilizes soft-attention interactions to achieve cross-modal feature perception from both consistency and inconsistency perspectives, where we also design the interaction constraints to ensure the interaction quality. Extensive experiments demonstrate the superior performance of our FMS compared to state-of-the-art methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
