arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.16754cs.CVcs.AI
收藏

视觉反图灵测试(VCT2):评估AI生成图像检测的基准与视觉AI指数(VAI)

The Visual Counter Turing Test (VCT2): A Benchmark for Evaluating AI-Generated Image Detection and the Visual AI Index (VAI)

University of South Carolina(南卡罗来纳大学);Manipal University Jaipur(贾伊普尔Manipal大学);BITS Pilani(比斯-皮尔尼大学);Xpectrum AI(Xpectrum人工智能);International Institute of Information Technology(国际信息科技研究所);Texas A&M University(德克萨斯农工大学);Carnegie Mellon University(卡内基梅隆大学);Vishwakarma Institute of Information Technology(维什瓦克arma信息科技研究所);IIIT Delhi(德里IIIT);National Institute of Technology, Silchar(锡拉char国家理工学院);Amazon AI(亚马逊人工智能);Stanford University(斯坦福大学);Amazon GenAI(亚马逊生成人工智能);Shenzhen University of Advanced Technology(深圳先进技术大学);BITS Pilani, Goa(果阿BITS Pilani大学)
Nasrin Imanpour, Abhilekh Borah, Shashwat Bajpai, Subhankar Ghosh, Sainath Reddy Sankepally, Hasnat Md Abdullah, Nishoak Kosaraju, Shreyas Dixit, Ashhar Aziz, S… 展开作者

Nasrin Imanpour, Abhilekh Borah, Shashwat Bajpai, Subhankar Ghosh, Sainath Reddy Sankepally, Hasnat Md Abdullah, Nishoak Kosaraju, Shreyas Dixit, Ashhar Aziz, Shwetangshu Biswas, Vinija Jain, Aman Chadha, Song Wang, Amit Sheth, Amitava Das

AI总结:

针对AGID方法过拟合已知生成器的问题,本文提出VCT2基准(含16.6万图像、6种T2I模型生成数据)与VAI指数,零样本测试显示现有AGID模型准确率仅约58%,VAI与检测准确率负相关,相关资源已开源以促进研究。

摘要:

文本到图像(T2I)生成模型的快速发展和广泛普及,加剧了人们对AI生成视觉内容被滥用的担忧,尤其是在错误信息传播活动中。现有的AI生成图像检测(AGID)方法往往过拟合于已知生成器,在面对较新或未见过的模型输出时表现不佳。我们引入了视觉反图灵测试(VCT2),这是一个包含166,000张图像的综合基准,由真实图像和6个最先进T2I系统生成的合成提示-图像对组成:Stable Diffusion 2.1、SDXL、SD3 Medium、SD3.5 Large、DALL.E 3和Midjourney 6。我们构建了两个不同的子集:COCOAI,包含来自MS COCO的结构化描述;TwitterAI,包含来自《纽约时报》的叙事风格推文。在统一的零样本评估下,我们对17个领先的AGID模型进行了基准测试,发现检测准确率极低,COCOAI上为58%,TwitterAI上为58.34%。为了超越二分类,我们提出了视觉AI指数(VAI),这是一种基于12个低级视觉特征的可解释、与提示无关的真实感指标,使我们能够更细致地量化和排名生成输出的感知质量。相关性分析显示VAI与检测准确率之间存在中等程度的负相关:COCOAI上的皮尔逊系数为-0.532,TwitterAI上为-0.503,表明视觉上更真实的图像往往更难检测,这一趋势在所有生成器中一致存在。我们发布了COCOAI、TwitterAI和所有代码,以推动广义AGID和感知真实感评估的未来发展。

英文摘要:

The rapid progress and widespread availability of text-to-image (T2I) generative models have heightened concerns about the misuse of AI-generated visuals, particularly in the context of misinformation campaigns. Existing AI-generated image detection (AGID) methods often overfit to known generators and falter on outputs from newer or unseen models. We introduce the Visual Counter Turing Test (VCT2), a comprehensive benchmark of 166,000 images, comprising both real and synthetic prompt-image pairs produced by six state-of-the-art T2I systems: Stable Diffusion 2.1, SDXL, SD3 Medium, SD3.5 Large, DALL.E 3, and Midjourney 6. We curate two distinct subsets: COCOAI, featuring structured captions from MS COCO, and TwitterAI, containing narrative-style tweets from The New York Times. Under a unified zero-shot evaluation, we benchmark 17 leading AGID models and observe alarmingly low detection accuracy, 58% on COCOAI and 58.34% on TwitterAI. To transcend binary classification, we propose the Visual AI Index (VAI), an interpretable, prompt-agnostic realism metric based on twelve low-level visual features, enabling us to quantify and rank the perceptual quality of generated outputs with greater nuance. Correlation analysis reveals a moderate inverse relationship between VAI and detection accuracy: Pearson of -0.532 on COCOAI and -0.503 on TwitterAI, suggesting that more visually realistic images tend to be harder to detect, a trend observed consistently across generators. We release COCOAI, TwitterAI, and all codes to catalyze future advances in generalized AGID and perceptual realism assessment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑