arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.04470cs.CVcs.AI
收藏

DiffuSyn Bench: 评估视觉-语言模型在现实世界复杂性中的能力,通过扩散生成的合成基准

DiffuSyn Bench: Evaluating Vision-Language Models on Real-World Complexities with Diffusion-Generated Synthetic Benchmarks

Imperial College London(伦敦帝国理工学院);Shanghai University(上海大学)

Haokun Zhou, Yipeng Hong

AI总结:

DiffuSyn Bench通过扩散生成的合成基准评估视觉-语言模型在现实世界复杂性中的能力,揭示其区分AI与人类图像的性能及局限性。

摘要:

本研究评估了大型视觉-语言模型(LVLMs)区分人工智能生成图像和人类生成图像的能力。它介绍了一种新的自动化基准构建方法用于此评估。实验比较了常见LVLMs与人类参与者,使用混合的AI和人类生成图像数据集。结果显示,LVLMs在一定程度上能够区分图像类型,但表现出右偏倾向,并且在人类面前表现显著较差。为了进一步验证这些发现,我们开发了一种使用AI自动构建基准的过程。该过程包括主题检索、叙述脚本生成、错误嵌入和图像生成,创建了一组具有故意错误的文本-图像对。我们通过构建两个可扩展的基准来验证我们的方法。本研究突显了LVLMs在现实世界理解中的优势和劣势,并推进了基准构建技术,提供了一种可扩展和自动化的AI模型评估方法。

英文摘要:

This study assesses the ability of Large Vision-Language Models (LVLMs) to differentiate between AI-generated and human-generated images. It introduces a new automated benchmark construction method for this evaluation. The experiment compared common LVLMs with human participants using a mixed dataset of AI and human-created images. Results showed that LVLMs could distinguish between the image types to some extent but exhibited a rightward bias, and perform significantly worse compared to humans. To build on these findings, we developed an automated benchmark construction process using AI. This process involved topic retrieval, narrative script generation, error embedding, and image generation, creating a diverse set of text-image pairs with intentional errors. We validated our method through constructing two caparable benchmarks. This study highlights the strengths and weaknesses of LVLMs in real-world understanding and advances benchmark construction techniques, providing a scalable and automatic approach for AI model evaluation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑