视觉反图灵测试(VCT2):评估AI生成图像检测的基准与视觉AI指数(VAI)
The Visual Counter Turing Test (VCT2): A Benchmark for Evaluating AI-Generated Image Detection and the Visual AI Index (VAI)
AI总结:
针对AGID方法过拟合已知生成器的问题,本文提出VCT2基准(含16.6万图像、6种T2I模型生成数据)与VAI指数,零样本测试显示现有AGID模型准确率仅约58%,VAI与检测准确率负相关,相关资源已开源以促进研究。
摘要:
文本到图像(T2I)生成模型的快速发展和广泛普及,加剧了人们对AI生成视觉内容被滥用的担忧,尤其是在错误信息传播活动中。现有的AI生成图像检测(AGID)方法往往过拟合于已知生成器,在面对较新或未见过的模型输出时表现不佳。我们引入了视觉反图灵测试(VCT2),这是一个包含166,000张图像的综合基准,由真实图像和6个最先进T2I系统生成的合成提示-图像对组成:Stable Diffusion 2.1、SDXL、SD3 Medium、SD3.5 Large、DALL.E 3和Midjourney 6。我们构建了两个不同的子集:COCOAI,包含来自MS COCO的结构化描述;TwitterAI,包含来自《纽约时报》的叙事风格推文。在统一的零样本评估下,我们对17个领先的AGID模型进行了基准测试,发现检测准确率极低,COCOAI上为58%,TwitterAI上为58.34%。为了超越二分类,我们提出了视觉AI指数(VAI),这是一种基于12个低级视觉特征的可解释、与提示无关的真实感指标,使我们能够更细致地量化和排名生成输出的感知质量。相关性分析显示VAI与检测准确率之间存在中等程度的负相关:COCOAI上的皮尔逊系数为-0.532,TwitterAI上为-0.503,表明视觉上更真实的图像往往更难检测,这一趋势在所有生成器中一致存在。我们发布了COCOAI、TwitterAI和所有代码,以推动广义AGID和感知真实感评估的未来发展。
英文摘要:
The rapid progress and widespread availability of text-to-image (T2I) generative models have heightened concerns about the misuse of AI-generated visuals, particularly in the context of misinformation campaigns. Existing AI-generated image detection (AGID) methods often overfit to known generators and falter on outputs from newer or unseen models. We introduce the Visual Counter Turing Test (VCT2), a comprehensive benchmark of 166,000 images, comprising both real and synthetic prompt-image pairs produced by six state-of-the-art T2I systems: Stable Diffusion 2.1, SDXL, SD3 Medium, SD3.5 Large, DALL.E 3, and Midjourney 6. We curate two distinct subsets: COCOAI, featuring structured captions from MS COCO, and TwitterAI, containing narrative-style tweets from The New York Times. Under a unified zero-shot evaluation, we benchmark 17 leading AGID models and observe alarmingly low detection accuracy, 58% on COCOAI and 58.34% on TwitterAI. To transcend binary classification, we propose the Visual AI Index (VAI), an interpretable, prompt-agnostic realism metric based on twelve low-level visual features, enabling us to quantify and rank the perceptual quality of generated outputs with greater nuance. Correlation analysis reveals a moderate inverse relationship between VAI and detection accuracy: Pearson of -0.532 on COCOAI and -0.503 on TwitterAI, suggesting that more visually realistic images tend to be harder to detect, a trend observed consistently across generators. We release COCOAI, TwitterAI, and all codes to catalyze future advances in generalized AGID and perceptual realism assessment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。