arXiv 2211.086152022-11cs.CV

GLFF：用于AI合成图像检测的全局与局部特征融合

GLFF: Global and Local Feature Fusion for AI-synthesized Image Detection

University at Buffalo (UB), State University of New York（纽约州立大学布法罗分校）；National Institute of Standards and Technology (NIST)（美国国家标准与技术研究院）

Yan Ju, Shan Jia, Jialing Cai, Haiying Guan, Siwei Lyu

备注：: 13 pages, 6 figures, 8 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对AI合成图像检测在现实场景中性能下降的问题，提出GLFF框架，融合全局多尺度特征与局部精细伪影特征，并构建含6种生成模型及多种后处理的DF 3数据集，实验验证其优越性。

摘要：

随着深度生成模型（如生成对抗网络和扩散模型）的快速发展，AI合成图像的质量已高到人类难以将其与原始图像区分开来的程度。尽管现有检测方法在特定评估设置（例如，针对已知模型生成的图像或未经真实世界后处理的图像）中表现出高性能，但在现实场景中，当测试图像可能由更强大的生成模型生成或与各种后处理操作相结合时，它们往往遭受严重的性能下降。为了解决这一问题，我们提出了一个全局与局部特征融合（GLFF）框架，通过结合来自整个图像的多尺度全局特征与来自信息丰富补丁的精细局部特征，学习丰富且具有判别性的表示，用于AI合成图像检测。GLFF融合了两个分支的信息：全局分支用于提取多尺度语义特征，局部分支用于选择信息丰富的补丁以提取详细的局部伪影。由于缺乏模拟现实世界应用的合成图像数据集进行评估，我们进一步创建了一个具有挑战性的伪造图像数据集，名为DeepFakeFaceForensics（DF 3），该数据集包含6种最先进的生成模型和多种后处理技术，以接近现实世界场景。实验结果表明，在我们提出的DF 3数据集和其他三个开源数据集上，我们的方法优于最先进的方法。

英文摘要：

With the rapid development of deep generative models (such as Generative Adversarial Networks and Diffusion models), AI-synthesized images are now of such high quality that humans can hardly distinguish them from pristine ones. Although existing detection methods have shown high performance in specific evaluation settings, e.g., on images from seen models or on images without real-world post-processing, they tend to suffer serious performance degradation in real-world scenarios where testing images can be generated by more powerful generation models or combined with various post-processing operations. To address this issue, we propose a Global and Local Feature Fusion (GLFF) framework to learn rich and discriminative representations by combining multi-scale global features from the whole image with refined local features from informative patches for AI synthesized image detection. GLFF fuses information from two branches: the global branch to extract multi-scale semantic features and the local branch to select informative patches for detailed local artifacts extraction. Due to the lack of a synthesized image dataset simulating real-world applications for evaluation, we further create a challenging fake image dataset, named DeepFakeFaceForensics (DF 3 ), which contains 6 state-of-the-art generation models and a variety of post-processing techniques to approach the real-world scenarios. Experimental results demonstrate the superiority of our method to the state-of-the-art methods on the proposed DF 3 dataset and three other open-source datasets.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文