GIE-Bench：迈向文本引导图像编辑的落地评估
GIE-Bench: Towards Grounded Evaluation for Text-Guided Image Editing
AI总结：
本文提出 GIE-Bench 基准，通过多项选择题评估功能正确性，并结合对象感知掩码与保持评分评估非目标区域保持，从而更可靠地评测文本引导图像编辑模型。
摘要：
使用自然语言指令编辑图像已成为修改视觉内容的一种自然且富有表现力的方式；然而，评估此类模型的性能仍具挑战。现有评估方法通常依赖 CLIP 等图文相似度指标，缺乏精确性。在本工作中，我们引入了一个新基准，旨在以更具落地依据的方式评估文本引导图像编辑模型，涵盖两个关键维度：(i) 功能正确性，通过自动生成的多项选择题进行评估，以验证预期更改是否成功应用；(ii) 图像内容保持，使用对象感知掩码技术和保持评分来确保图像的非目标区域在视觉上保持一致。该基准包含跨越 20 个不同内容类别的 1000 多个高质量编辑示例，每个示例都标注了详细的编辑指令、评估问题和空间对象掩码。我们开展了一项大规模研究，将文本引导图像编辑领域的最新旗舰模型 GPT-Image-1 与若干最先进的编辑模型进行比较，并对照人工评分验证了我们的自动指标。结果表明，GPT-Image-1 在指令遵循准确率方面领先，但往往会过度修改无关图像区域，这凸显了当前模型行为中的一个关键权衡。GIE-Bench 为推进更准确的文本引导图像编辑评估提供了一个可扩展、可复现的框架。
英文摘要：
Editing images using natural language instructions has become a natural and expressive way to modify visual content; yet, evaluating the performance of such models remains challenging. Existing evaluation approaches often rely on image-text similarity metrics like CLIP, which lack precision. In this work, we introduce a new benchmark designed to evaluate text-guided image editing models in a more grounded manner, along two critical dimensions: (i) functional correctness, assessed via automatically generated multiple-choice questions that verify whether the intended change was successfully applied; and (ii) image content preservation, which ensures that non-targeted regions of the image remain visually consistent using an object-aware masking technique and preservation scoring. The benchmark includes over 1000 high-quality editing examples across 20 diverse content categories, each annotated with detailed editing instructions, evaluation questions, and spatial object masks. We conduct a large-scale study comparing GPT-Image-1, the latest flagship in the text-guided image editing space, against several state-of-the-art editing models, and validate our automatic metrics against human ratings. Results show that GPT-Image-1 leads in instruction-following accuracy, but often over-modifies irrelevant image regions, highlighting a key trade-off in the current model behavior. GIE-Bench provides a scalable, reproducible framework for advancing more accurate evaluation of text-guided image editing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
