arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.01790cs.CV
收藏

MMFusion:结合图像取证滤波器用于视觉篡改检测与定位

MMFusion: Combining Image Forensic Filters for Visual Manipulation Detection and Localization

Centre for Research and Technology Hellas (CERTH)(希腊研究与技术中心(CERTH));Information Technologies Institute (ITI)(信息技术研究所(ITI));Department of Computer Science, Stony Brook University(石溪大学计算机科学系)

Kostas Triaridis, Konstantinos Tsigos, Vasileios Mezaris

AI总结:

本文提出 MMFusion 架构,通过早期融合多种图像取证滤波器的互补痕迹并引入特征重加权解码机制,在多个图像和视频数据集上实现了优于现有方法的篡改检测与定位性能。

摘要:

近年来,图像篡改定位与检测技术通常利用由噪声敏感滤波器(如 SRM 或 Bayar 卷积)产生的取证伪影和痕迹。本文展示了此类方法中常用的不同滤波器在揭示不同类型的篡改方面各有所长,并能提供互补的取证痕迹。因此,我们探索了组合这些滤波器输出的方法,以利用所产生伪影的互补性来执行图像篡改定位与检测(IMLD)。我们评估了两种不同的组合方法:一种是从每个取证滤波器生成独立特征后再进行融合(称为晚期融合),另一种是对不同模态输出进行早期混合并生成组合特征(称为早期融合)。我们将后者用作特征编码机制,并配以包含特征重加权的新解码机制,构建了所提出的 MMFusion 架构。我们证明,MMFusion 在图像篡改定位和检测方面均取得了具有竞争力的性能,在多个图像和视频数据集上超越了最先进的模型。我们还基于近期的人工智能可解释性方法,进一步研究了 MMFusion 中每个取证滤波器在处理不同类型篡改时的贡献。

英文摘要:

Recent image manipulation localization and detection techniques typically leverage forensic artifacts and traces that are produced by a noise-sensitive filter, such as SRM or Bayar convolution. In this paper, we showcase that different filters commonly used in such approaches excel at unveiling different types of manipulations and provide complementary forensic traces. Thus, we explore ways of combining the outputs of such filters to leverage the complementary nature of the produced artifacts for performing image manipulation localization and detection (IMLD). We assess two distinct combination methods: one that produces independent features from each forensic filter and then fuses them (this is referred to as late fusion) and one that performs early mixing of different modal outputs and produces combined features (this is referred to as early fusion). We use the latter as a feature encoding mechanism, accompanied by a new decoding mechanism that encompasses feature re-weighting, for formulating the proposed MMFusion architecture. We demonstrate that MMFusion achieves competitive performance for both image manipulation localization and detection, outperforming state-of-the-art models across several image and video datasets. We also investigate further the contribution of each forensic filter within MMFusion for addressing different types of manipulations, building on recent AI explainability measures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑