arXiv 2605.189842026-05-18cs.CV

Artifact-Bench: 评估MLLMs在检测和评估AI生成视频中的伪影

Artifact-Bench: Evaluating MLLMs on Detecting and Assessing the Artifacts of AI-Generated Videos

Kling Team（Kling团队）；Shanghai AI Lab（上海人工智能实验室）

Yuqi Tang, Yang Shi, Zhuoran Zhang, Qixun Wang, Xuehai Bai, Yue Ding, Ruizhe Chen, Bohan Zeng, Xinlong Chen, Xuanyu Zhu, Bozhou Li, Yuran Wang, Yifan Dai, Cheng… 展开作者

Yuqi Tang, Yang Shi, Zhuoran Zhang, Qixun Wang, Xuehai Bai, Yue Ding, Ruizhe Chen, Bohan Zeng, Xinlong Chen, Xuanyu Zhu, Bozhou Li, Yuran Wang, Yifan Dai, Chengzhuo Tong, Xinyu Liu, Yiyan Ji, Yujie Wei, Yuhao Dong, Shilin Yan, Fengxiang Wang, Yi-Fan Zhang, Haotian Wang, Yuanxing Zhang, Pengfei Wan

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出Artifact-Bench，一个用于评估多模态大语言模型在检测和分析AI生成视频伪影能力的基准，揭示了现有模型在伪影感知和推理上的显著局限性。

摘要：

近年来，视频生成模型在提高AI生成视频的真实感方面取得了显著进步，但其输出仍存在时间不一致、结构失真和语义不连贯等伪影。尽管多模态大语言模型（MLLMs）在视觉理解方面表现出色，但其感知和推理这些伪影的能力仍不明确。现有基准缺乏对伪影感知和细粒度诊断推理的系统评估，尤其是在超越逼真内容的多样化AI生成视频领域。为解决这一差距，我们引入Artifact-Bench，一个全面的基准，用于评估MLLMs在AI生成视频伪影检测和分析上的能力。我们首先建立了涵盖逼真、动画和CG风格视频的三级层次化伪影分类法。基于此分类法，Artifact-Bench定义了三个互补任务：真实与AI生成视频分类、成对真实感比较和细粒度伪影识别。在19种领先MLLMs上的实验揭示了伪影感知和推理的显著局限性，许多模型在挑战性设置中接近随机甚至低于随机表现。我们进一步观察到MLLM判断与人类感知偏好之间存在显著不一致，突显了其作为AI生成视频真实感一般评估者的有限可靠性。

英文摘要：

Recent video generative models have greatly improved the realism of AI-generated videos, yet their outputs still exhibit artifacts such as temporal inconsistencies, structural distortions, and semantic incoherence. While Multimodal Large Language Models (MLLMs) show strong visual understanding capabilities, their ability to perceive and reason about such artifacts remains unclear. Existing benchmarks often lack systematic evaluation of artifact-aware perception and fine-grained diagnostic reasoning, especially across diverse AI-generated video domains beyond photorealistic content. To address this gap, we introduce Artifact-Bench, a comprehensive benchmark for evaluating MLLMs on AI-generated video artifact detection and analysis. We first establish a three-level hierarchical taxonomy of realism artifacts, covering photorealistic, animated, and CG-style videos. Based on this taxonomy, Artifact-Bench defines three complementary tasks: real vs. AI-generated video classification, pairwise realism comparison, and fine-grained artifact identification. Experiments on 19 leading MLLMs reveal substantial limitations in artifact perception and reasoning, with many models approaching random or even below-random performance in challenging settings. We further observe significant misalignment between MLLM judgments and human perceptual preferences, highlighting their limited reliability as general evaluators for AI-generated video realism.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。