SalArt-VQA: 诊断VLM是否理解生成图像中的显著伪影
SalArt-VQA: Diagnosing Whether VLMs Understand Salient Artifacts in Generated Images
AI总结：
提出SalArt-VQA基准，通过950张图像和3681道多选题，从检测、定位、空间基础、缺陷识别四方面评估VLM对生成图像伪影的理解，揭示高检测准确率下隐藏的失败模式。
摘要：
视觉语言模型（VLM）越来越多地被用于检测AI生成图像是否包含可见伪影，然而它们分析此类伪影的能力仍然知之甚少。正确的图像级决策仍可能隐藏重要失败：模型可能正确标记伪影，但依赖于错误的视觉线索、选择错误的区域，或描述图像中不存在的缺陷。为了直接评估这些行为，我们引入了SalArt-VQA，一个用于细粒度理解AI生成图像中显著伪影的诊断基准。SalArt-VQA包含950张图像和3,681道人工编写的多项选择题，涵盖伪影图像、匹配的真实参考图像和配对的生成参考图像。四种对齐的问题类型评估存在检测、语义定位、空间基础和证据基础的缺陷识别，而参考分割测试了当注释缺陷不存在时的校准和弃权能力。在20个VLM上，SalArt-VQA揭示了图像级检测准确率所隐藏的失败：最强的模型在伪影图像上达到99.37%的检测召回率，但仅在53.26%的图像上正确回答了所有四个伪影侧问题。比较伪影图像与无伪影参考揭示了灵敏度-校准权衡：敏感模型经常做出无根据的伪影声明，而保守模型主要通过遗漏真实伪影来避免误报。这些结果表明，高伪影检测准确率本身并不意味着有基础的伪影理解。SalArt-VQA暴露了这些隐藏的失败模式，并提供了对VLM伪影声明是否得到局部视觉证据支持的细粒度评估。
英文摘要：
Vision-language models (VLMs) are increasingly used to detect whether AI-generated images contain visible artifacts, yet their ability to analyze such artifacts remains poorly understood. A correct image-level decision can still hide important failures: a model may correctly flag an artifact while relying on the wrong visual cue, selecting the wrong region, or describing a defect that the image does not support. To evaluate these behaviors directly, we introduce SalArt-VQA, a diagnostic benchmark for fine-grained SALient ARTifact understanding in AI-generated images. SalArt-VQA contains 950 images and 3,681 human-authored multiple-choice questions spanning artifact images, matched real reference images, and paired generated reference images. Four aligned question types evaluate presence detection, semantic localization, spatial grounding, and evidence-grounded defect identification, while the reference splits test calibration and abstention when the annotated defect is absent. Across 20 VLMs, SalArt-VQA reveals failures that image-level detection accuracy hides: the strongest model reaches 99.37% detection recall on artifact images but answers all four artifact-side questions correctly on only 53.26% of images. Comparing artifact images with artifact-free references reveals a sensitivity-calibration tradeoff: sensitive models often make unsupported artifact claims, while conservative models avoid false alarms largely by missing real artifacts. These results show that high artifact detection accuracy alone does not imply grounded artifact understanding. SalArt-VQA exposes these hidden failure modes and provides a fine-grained evaluation of whether VLM artifact claims are supported by local visual evidence.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
