FakeOut：利用域外自监督进行多模态视频深度伪造检测
FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection
AI总结：
FakeOut利用多模态域外自监督骨干网络，适应视频深度伪造检测，在跨数据集泛化上达到最先进结果，证明域外训练可提升检测鲁棒性。
摘要：
近年来，视频合成方法迅速改进，使得创建合成人类变得轻而易举。这带来了一个问题，尤其是在社交媒体时代，因为说话人类的合成视频可以被用来以令人信服的方式传播错误信息。因此，迫切需要准确且鲁棒的深度伪造检测方法，能够检测训练中未见过的伪造技术。在这项工作中，我们探讨是否可以通过利用一个以自监督方式训练的、多模态的、域外骨干网络，并使其适应视频深度伪造领域来实现这一目标。我们提出了FakeOut；一种新颖的方法，在预训练阶段和适应阶段都依赖多模态数据。我们证明了FakeOut在检测各种类型的深度伪造，尤其是训练中未见过的操作方面的有效性和鲁棒性。我们的方法在音视频数据集的跨数据集泛化上达到了最先进的结果。这项研究表明，也许令人惊讶的是，在域外视频（即并非特别以说话人类为特征）上训练，可以带来更好的深度伪造检测系统。代码可在GitHub上获取。
英文摘要：
Video synthesis methods rapidly improved in recent years, allowing easy creation of synthetic humans. This poses a problem, especially in the era of social media, as synthetic videos of speaking humans can be used to spread misinformation in a convincing manner. Thus, there is a pressing need for accurate and robust deepfake detection methods, that can detect forgery techniques not seen during training. In this work, we explore whether this can be done by leveraging a multi-modal, out-of-domain backbone trained in a self-supervised manner, adapted to the video deepfake domain. We propose FakeOut; a novel approach that relies on multi-modal data throughout both the pre-training phase and the adaption phase. We demonstrate the efficacy and robustness of FakeOut in detecting various types of deepfakes, especially manipulations which were not seen during training. Our method achieves state-of-the-art results in cross-dataset generalization on audio-visual datasets. This study shows that, perhaps surprisingly, training on out-of-domain videos (i.e., not especially featuring speaking humans), can lead to better deepfake detection systems. Code is available on GitHub.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
