arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.02733cs.CV
收藏

MMNet:面向序列化深度伪造检测的多协作与多监督网络

MMNet: Multi-Collaboration and Multi-Supervision Network for Sequential Deepfake Detection

Xidian University(西安电子科技大学);Chongqing University of Posts and Telecommunications(重庆邮电大学)

Ruiyang Xia, Decheng Liu, Jie Li, Lin Yuan, Nannan Wang, Xinbo Gao

AI总结:

本文提出 MMNet,通过多协作与多监督机制处理伪造人脸图像中的空间尺度和序列排列差异,无需已知操纵方法即可实现序列化深度伪造检测与恢复,并设计 CSM 指标评估多步检测匹配度,在多个数据集上取得领先性能。

摘要:

先进的操纵技术为犯罪分子提供了通过生成欺骗性媒体(例如伪造人脸图像)制造社会恐慌或获取非法利益的机会。为此,各种深度伪造检测方法被提出以评估图像真实性。序列化深度伪造检测作为深度伪造检测的延伸,旨在以正确的顺序识别伪造面部区域以进行恢复。然而,由于空间操纵和序列操纵的不同组合,伪造人脸图像表现出显著差异,严重影响了检测性能。此外,伪造图像的恢复需要了解操纵模型以执行逆变换,但由于相关技术常被攻击者隐藏,这一点难以确定。为解决这些问题,我们提出了多协作与多监督网络(MMNet),该网络能够处理伪造人脸图像中的各种空间尺度和序列排列,并在无需了解相应操纵方法的情况下实现恢复。此外,现有评估指标仅考虑单个推断步骤下的检测准确率,未考虑连续多个步骤下与真值的匹配程度。为克服这一局限,我们提出了一种新的评估指标,称为完整序列匹配(CSM),该指标考虑多个推断步骤下的检测准确率,反映检测完整伪造序列的能力。在多个典型数据集上的大量实验表明,MMNet 实现了最先进的检测性能和独立的恢复性能。

英文摘要:

Advanced manipulation techniques have provided criminals with opportunities to make social panic or gain illicit profits through the generation of deceptive media, such as forged face images. In response, various deepfake detection methods have been proposed to assess image authenticity. Sequential deepfake detection, which is an extension of deepfake detection, aims to identify forged facial regions with the correct sequence for recovery. Nonetheless, due to the different combinations of spatial and sequential manipulations, forged face images exhibit substantial discrepancies that severely impact detection performance. Additionally, the recovery of forged images requires knowledge of the manipulation model to implement inverse transformations, which is difficult to ascertain as relevant techniques are often concealed by attackers. To address these issues, we propose Multi-Collaboration and Multi-Supervision Network (MMNet) that handles various spatial scales and sequential permutations in forged face images and achieve recovery without requiring knowledge of the corresponding manipulation method. Furthermore, existing evaluation metrics only consider detection accuracy at a single inferring step, without accounting for the matching degree with ground-truth under continuous multiple steps. To overcome this limitation, we propose a novel evaluation metric called Complete Sequence Matching (CSM), which considers the detection accuracy at multiple inferring steps, reflecting the ability to detect integrally forged sequences. Extensive experiments on several typical datasets demonstrate that MMNet achieves state-of-the-art detection performance and independent recovery performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑