arXiv 2406.165312025-01-13cs.CV

GIM：面向生成式图像篡改检测与定位的百万级基准

GIM: A Million-scale Benchmark for Generative Image Manipulation Detection and Localization

Shanghai Jiao Tong University（上海交通大学）；Tsinghua University（清华大学）

Yirui Chen, Xudong Huang, Quan Zhang, Wei Li, Mingjian Zhu, Qiangyu Yan, Simiao Li, Hanting Chen, Hailin Hu, Jie Yang, Wei Liu, Jie Hu

备注：: Code page: https://github.com/chenyirui/GIM

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对图像篡改检测与定位任务缺乏大规模数据的问题，本文构建数据生成流水线，推出百万级GIM数据集与基准，并提出GIMFormer框架，实验证明其性能超越现有最优方法。

摘要：

生成模型的卓越能力成为图像编辑与生成逼真图像的新趋势，对多媒体数据的可信性构成严重威胁，也推动了图像篡改检测与定位（IMDL）的研究。然而，大规模数据基础的缺失使得IMDL任务的深入推进受限。本文构建了一套融合SAM（分割一切模型）、LLM（大语言模型）与生成模型强大能力的局部篡改数据生成流水线。在此基础上，我们提出GIM数据集，它具备以下优势：1）规模大，GIM包含超过一百万对AI篡改图像与真实图像；2）图像内容丰富，涵盖广泛的图像类别；3）生成式篡改类型多样，图像均由最先进的生成器制作，涉及多种篡改任务。上述优势使得能够对IMDL方法开展更全面的评估，拓展其在不同图像上的适用性。我们引入设置了两种评测模式的GIM基准，用于评估现有IMDL方法。此外，我们提出一种新型IMDL框架，名为GIMFormer，它由ShadowTracer、频率-空间模块（FSB）和多窗口异常建模（MWAM）模块组成。在GIM上开展的大量实验表明，GIMFormer在两个不同基准上均超越了此前的最优方法。

英文摘要：

The extraordinary ability of generative models emerges as a new trend in image editing and generating realistic images, posing a serious threat to the trustworthiness of multimedia data and driving the research of image manipulation detection and location (IMDL). However, the lack of a large-scale data foundation makes the IMDL task unattainable. In this paper, we build a local manipulation data generation pipeline that integrates the powerful capabilities of SAM, LLM, and generative models. Upon this basis, we propose the GIM dataset, which has the following advantages: 1) Large scale, GIM includes over one million pairs of AI-manipulated images and real images. 2) Rich image content, GIM encompasses a broad range of image classes. 3) Diverse generative manipulation, the images are manipulated images with state-of-the-art generators and various manipulation tasks. The aforementioned advantages allow for a more comprehensive evaluation of IMDL methods, extending their applicability to diverse images. We introduce the GIM benchmark with two settings to evaluate existing IMDL methods. In addition, we propose a novel IMDL framework, termed GIMFormer, which consists of a ShadowTracer, Frequency-Spatial block (FSB), and a Multi-Window Anomalous Modeling (MWAM) module. Extensive experiments on the GIM demonstrate that GIMFormer surpasses the previous state-of-the-art approach on two different benchmarks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。