鲁棒的序列DeepFake检测
Robust Sequential DeepFake Detection
AI总结：
针对现有DeepFake检测仅支持单步操控二分类的局限，该研究提出序列DeepFake操控检测新任务，构建了首个对应数据集及扰动版本，并设计两款Transformer模型实现鲁棒的序列操控预测。
摘要：
如今，人脸操控技术可轻松生成照片级真实感的人脸，这类技术的潜在恶意滥用引发了广泛关注，因此众多DeepFake检测方法被相继提出。然而，现有方法仅聚焦于检测单步人脸操控。随着易于获取的人脸编辑应用不断涌现，人们能够通过多步操作以序列方式轻松修改人脸组件。这一新威胁要求我们能够检测一系列人脸操控操作，这对于DeepFake媒体检测以及后续的原始人脸恢复都至关重要。受此观察启发，我们强调了相关研究的必要性，并提出了一个名为Detecting Sequential DeepFake Manipulation（Seq-DeepFake，序列DeepFake操控检测）的全新研究问题。现有DeepFake检测任务仅需预测二值标签，而Seq-DeepFake操控检测则需要正确预测人脸操控操作的序列向量。为支撑大规模研究，我们构建了首个Seq-DeepFake数据集，其中的人脸图像经过序列操控，并标注有对应的人脸操控序列向量。基于该新数据集，我们将Seq-DeepFake操控检测建模为一种特定的图像到序列任务，并提出了简洁且有效的Seq-DeepFake Transformer（SeqFakeFormer）。为更贴近真实世界的DeepFake数据分布，我们进一步在原始Seq-DeepFake数据集上施加多种扰动，构建了更具挑战性的带扰动序列DeepFake数据集（Seq-DeepFake-P）。为在面对Seq-DeepFake-P时挖掘图像与序列之间更深层的关联，我们设计了专用的带图像-序列推理的Seq-DeepFake Transformer（SeqFakeFormer++），该模型能在图像-序列对之间建立更强的对应关系，以实现更鲁棒的Seq-DeepFake检测。
英文摘要：
Since photorealistic faces can be readily generated by facial manipulation technologies nowadays, potential malicious abuse of these technologies has drawn great concerns. Numerous deepfake detection methods are thus proposed. However, existing methods only focus on detecting one-step facial manipulation. As the emergence of easy-accessible facial editing applications, people can easily manipulate facial components using multi-step operations in a sequential manner. This new threat requires us to detect a sequence of facial manipulations, which is vital for both detecting deepfake media and recovering original faces afterwards. Motivated by this observation, we emphasize the need and propose a novel research problem called Detecting Sequential DeepFake Manipulation (Seq-DeepFake). Unlike the existing deepfake detection task only demanding a binary label prediction, detecting Seq-DeepFake manipulation requires correctly predicting a sequential vector of facial manipulation operations. To support a large-scale investigation, we construct the first Seq-DeepFake dataset, where face images are manipulated sequentially with corresponding annotations of sequential facial manipulation vectors. Based on this new dataset, we cast detecting Seq-DeepFake manipulation as a specific image-to-sequence task and propose a concise yet effective Seq-DeepFake Transformer (SeqFakeFormer). To better reflect real-world deepfake data distributions, we further apply various perturbations on the original Seq-DeepFake dataset and construct the more challenging Sequential DeepFake dataset with perturbations (Seq-DeepFake-P). To exploit deeper correlation between images and sequences when facing Seq-DeepFake-P, a dedicated Seq-DeepFake Transformer with Image-Sequence Reasoning (SeqFakeFormer++) is devised, which builds stronger correspondence between image-sequence pairs for more robust Seq-DeepFake detection.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
