ImagenWorld: 通过可解释的人类评估对图像生成模型进行压力测试，针对开放性现实任务
ImagenWorld: Stress-Testing Image Generation Models with Explainable Human Evaluation on Open-ended Real-World Tasks
AI总结：
ImagenWorld通过3600个条件集评估14个模型，揭示编辑任务比生成任务更难，艺术和照片实感领域表现优异，但符号和文本密集领域表现较差，现代VLM指标在Kendall准确性上达0.79，但无法实现细粒度可解释性误差归因。
摘要：
扩散、自回归和混合模型的进步使文本到图像、编辑和参考引导组合等任务的高质量图像合成成为可能。然而，现有基准仍然有限，要么专注于孤立任务，要么只覆盖狭窄领域，或提供不透明的评分而无失败模式解释。我们引入ImagenWorld，一个包含3600个条件集的基准，涵盖六个核心任务（生成和编辑，单或多参考）和六个主题领域（艺术品、逼真图像、信息图表、文本图表、计算机图形和截图）。该基准由20000个细粒度人工注释和可解释的评估方案支持，后者标记局部对象级和分段级错误，补充基于自动VLM的指标。我们对14个模型的大规模评估得出几个见解：（1）模型在编辑任务中比在生成任务中更困难，尤其是局部编辑。 （2）模型在艺术和逼真图像设置中表现优异，但在符号和文本密集领域如截图和信息图表中表现较差。 （3）封闭源系统总体领先，而针对数据编纂（如Qwen-Image）在文本密集情况下缩小了差距。 （4）现代基于VLM的指标在Kendall准确性上达到0.79，接近人类排名，但无法实现细粒度可解释性误差归因。 ImagenWorld提供了严格的基准和诊断工具，以推进稳健的图像生成。
英文摘要：
Advances in diffusion, autoregressive, and hybrid models have enabled high-quality image synthesis for tasks such as text-to-image, editing, and reference-guided composition. Yet, existing benchmarks remain limited, either focus on isolated tasks, cover only narrow domains, or provide opaque scores without explaining failure modes. We introduce \textbf{ImagenWorld}, a benchmark of 3.6K condition sets spanning six core tasks (generation and editing, with single or multiple references) and six topical domains (artworks, photorealistic images, information graphics, textual graphics, computer graphics, and screenshots). The benchmark is supported by 20K fine-grained human annotations and an explainable evaluation schema that tags localized object-level and segment-level errors, complementing automated VLM-based metrics. Our large-scale evaluation of 14 models yields several insights: (1) models typically struggle more in editing tasks than in generation tasks, especially in local edits. (2) models excel in artistic and photorealistic settings but struggle with symbolic and text-heavy domains such as screenshots and information graphics. (3) closed-source systems lead overall, while targeted data curation (e.g., Qwen-Image) narrows the gap in text-heavy cases. (4) modern VLM-based metrics achieve Kendall accuracies up to 0.79, approximating human ranking, but fall short of fine-grained, explainable error attribution. ImagenWorld provides both a rigorous benchmark and a diagnostic tool to advance robust image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
