GRADE: 图像编辑中学科引导推理的基准测试
GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing
AI总结：
GRADE是首个评估图像编辑中学科引导推理能力的基准测试，通过多领域样本和多维评估协议揭示当前模型在知识密集型编辑任务中的局限性。
摘要：
统一的多模态模型旨在实现联合理解、推理和生成，但当前图像编辑基准测试大多局限于自然图像和浅层常识推理，提供有限的评估能力，特别是在结构化、领域特定的约束下。在本工作中，我们介绍了GRADE，首个评估图像编辑中学科引导知识和推理的基准测试。GRADE包含520个精心挑选的样本，涵盖10个学术领域，从自然科学到社会科学。为了支持严谨的评估，我们提出了一种多维评估协议，共同评估学科推理、视觉一致性以及逻辑可读性。对20种最先进的开源和闭源模型进行的广泛实验揭示了当前模型在隐含、知识密集型编辑设置下的显著局限性，导致较大的性能差距。除了定量分数外，我们还进行了严谨的分析和消融实验，以揭示模型的不足并识别学科编辑中的约束。共同而言，GRADE指出了统一多模态模型未来发展的关键方向，推动了学科引导图像编辑和推理的研究。我们的基准和评估代码已公开发布。
英文摘要：
Unified multimodal models target joint understanding, reasoning, and generation, but current image editing benchmarks are largely confined to natural images and shallow commonsense reasoning, offering limited assessment of this capability under structured, domain-specific constraints. In this work, we introduce GRADE, the first benchmark to assess discipline-informed knowledge and reasoning in image editing. GRADE comprises 520 carefully curated samples across 10 academic domains, spanning from natural science to social science. To support rigorous evaluation, we propose a multi-dimensional evaluation protocol that jointly assesses Discipline Reasoning, Visual Consistency, and Logical Readability. Extensive experiments on 20 state-of-the-art open-source and closed-source models reveal substantial limitations in current models under implicit, knowledge-intensive editing settings, leading to large performance gaps. Beyond quantitative scores, we conduct rigorous analyses and ablations to expose model shortcomings and identify the constraints within disciplinary editing. Together, GRADE pinpoints key directions for the future development of unified multimodal models, advancing the research on discipline-informed image editing and reasoning. Our benchmark and evaluation code are publicly released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
