arXiv 2307.070362025-03-04cs.CV

GenConViT：使用生成式卷积视觉Transformer的深度伪造视频检测

GenConViT: Deepfake Video Detection Using Generative Convolutional Vision Transformer

Ghent University（根特大学）；imec；Addis Ababa University（亚的斯亚贝巴大学）；State University of New York Polytechnic Institute（纽约州立大学理工学院）

Deressa Wodajo Deressa, Hannes Mareen, Peter Lambert, Solomon Atnafu, Zahid Akhtar, Glenn Van Wallendael

备注：: 11 pages, 4 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出GenConViT模型，结合ConvNeXt和Swin Transformer提取特征，并利用自编码器和变分自编码器学习潜在分布，在多个数据集上实现了高准确率的深度伪造视频检测，但泛化能力仍需提升。

摘要：

深度伪造因其传播虚假信息并损害数字媒体完整性的潜在风险而引发了重大担忧。当前的深度伪造检测模型往往难以在各种深度伪造生成技术和视频内容之间进行泛化。在本研究中，我们提出了一种用于深度伪造视频检测的生成式卷积视觉Transformer（GenConViT）。我们的模型结合了ConvNeXt和Swin Transformer模型进行特征提取，并利用Autoencoder和Variational Autoencoder从潜在数据分布中学习。通过学习视觉伪影和潜在数据分布，GenConViT在检测各种深度伪造视频方面取得了更好的性能。该模型在DFDC、FF++、TM、DeepfakeTIMIT和Celeb-DF (v$2$)数据集上进行了训练和评估。提出的GenConViT模型在深度伪造视频检测中表现出强大的性能，在测试数据集上实现了高准确率。虽然我们的模型通过利用视觉和潜在特征在深度伪造视频检测中显示出有希望的结果，但我们证明仍需进一步工作来提高其泛化能力，即在遇到分布外数据时。我们的模型为识别各种伪造视频同时保持媒体完整性提供了有效的解决方案。GenConViT的开源代码可在https://github.com/erprogs/GenConViT获取。

英文摘要：

Deepfakes have raised significant concerns due to their potential to spread false information and compromise digital media integrity. Current deepfake detection models often struggle to generalize across a diverse range of deepfake generation techniques and video content. In this work, we propose a Generative Convolutional Vision Transformer (GenConViT) for deepfake video detection. Our model combines ConvNeXt and Swin Transformer models for feature extraction, and it utilizes Autoencoder and Variational Autoencoder to learn from the latent data distribution. By learning from the visual artifacts and latent data distribution, GenConViT achieves improved performance in detecting a wide range of deepfake videos. The model is trained and evaluated on DFDC, FF++, TM, DeepfakeTIMIT, and Celeb-DF (v$2$) datasets. The proposed GenConViT model demonstrates strong performance in deepfake video detection, achieving high accuracy across the tested datasets. While our model shows promising results in deepfake video detection by leveraging visual and latent features, we demonstrate that further work is needed to improve its generalizability, i.e., when encountering out-of-distribution data. Our model provides an effective solution for identifying a wide range of fake videos while preserving media integrity. The open-source code for GenConViT is available at https://github.com/erprogs/GenConViT.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。