面向可泛化深度伪造视频检测的漏洞感知时空学习
Vulnerability-Aware Spatio-Temporal Learning for Generalizable Deepfake Video Detection
AI总结：
本文提出FakeSTormer方法，通过多任务学习显式关注时空易伪影区域，并利用视频级数据合成生成细微伪影样本，以提升深度伪造视频检测的泛化能力。
摘要：
鉴于表征时空伪影的复杂性，检测深度伪造视频极具挑战性。现有大多数方法依赖于使用真实和伪造图像序列训练的二分类器，这阻碍了它们对未见生成方法的泛化能力。此外，随着生成式人工智能（AI）的不断进步，深度伪造伪影在空间和时间层面都变得难以察觉，使其极难被捕获。为解决这些问题，我们提出了一种名为FakeSTormer的细粒度深度伪造视频检测方法，该方法强化了对细微时空不一致性的建模，同时避免过拟合。具体而言，我们引入了一个多任务学习框架，其中包含两个辅助分支，用于显式关注易出现伪影的空间和时间区域。此外，我们提出了一种视频级数据合成策略，可生成具有细微时空伪影的伪伪造视频，为我们的附加分支提供高质量样本和免手工标注。在多个具有挑战性的基准上的大量实验表明，与近期最先进的方法相比，我们的方法具有优越性。代码可在 https://github.com/10Ring/FakeSTormer 获取。
英文摘要：
Detecting deepfake videos is highly challenging given the complexity of characterizing spatio-temporal artifacts. Most existing methods rely on binary classifiers trained using real and fake image sequences, therefore hindering their generalization capabilities to unseen generation methods. Moreover, with the constant progress in generative Artificial Intelligence (AI), deepfake artifacts are becoming imperceptible at both the spatial and the temporal levels, making them extremely difficult to capture. To address these issues, we propose a fine-grained deepfake video detection approach called FakeSTormer that enforces the modeling of subtle spatio-temporal inconsistencies while avoiding overfitting. Specifically, we introduce a multi-task learning framework that incorporates two auxiliary branches for explicitly attending artifact-prone spatial and temporal regions. Additionally, we propose a video-level data synthesis strategy that generates pseudo-fake videos with subtle spatio-temporal artifacts, providing high-quality samples and hand-free annotations for our additional branches. Extensive experiments on several challenging benchmarks demonstrate the superiority of our approach compared to recent state-of-the-art methods. The code is available at https://github.com/10Ring/FakeSTormer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
