I2EBench：面向基于指令的图像编辑的综合评测基准
I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing
AI总结：
本文提出 I2EBench，一个包含 2000 余张图像、4000 余条指令和 16 个评估维度的综合基准，用于自动评估基于指令的图像编辑模型，并通过用户研究对齐人类感知，为领域发展提供研究见解。
摘要：
基于指令的图像编辑（IIE）领域已取得显著进展，然而对这些模型的评估仍构成重大挑战。该领域的一个关键需求是建立综合评测基准，以准确评估编辑结果并为其进一步发展提供有价值的见解。为回应这一需求，我们提出 I2EBench，一个旨在从多个维度自动评估 IIE 模型所生成编辑图像质量的综合基准。I2EBench 包含 2,000 余张用于编辑的图像，以及 4,000 余条对应的原始且多样化的指令。它具备三个显著特点：1）综合评估维度：I2EBench 包含 16 个评估维度，覆盖高层和低层方面，能够对每个 IIE 模型进行全面评估。2）人类感知对齐：为确保基准与人类感知对齐，我们针对每个评估维度开展了大规模用户研究。3）有价值的研究见解：通过分析现有 IIE 模型在 16 个维度上的优势与不足，我们提供了有价值的研究见解，以指导该领域未来的发展。我们将开源 I2EBench，包括所有指令、输入图像、人类标注、所有被评估方法生成的编辑图像，以及用于评估新 IIE 模型结果的简单脚本。所有 IIE 模型的代码、数据集和生成图像均提供于 github：https://github.com/cocoshe/I2EBench。
英文摘要：
Significant progress has been made in the field of Instruction-based Image Editing (IIE). However, evaluating these models poses a significant challenge. A crucial requirement in this field is the establishment of a comprehensive evaluation benchmark for accurately assessing editing results and providing valuable insights for its further development. In response to this need, we propose I2EBench, a comprehensive benchmark designed to automatically evaluate the quality of edited images produced by IIE models from multiple dimensions. I2EBench consists of 2,000+ images for editing, along with 4,000+ corresponding original and diverse instructions. It offers three distinctive characteristics: 1) Comprehensive Evaluation Dimensions: I2EBench comprises 16 evaluation dimensions that cover both high-level and low-level aspects, providing a comprehensive assessment of each IIE model. 2) Human Perception Alignment: To ensure the alignment of our benchmark with human perception, we conducted an extensive user study for each evaluation dimension. 3) Valuable Research Insights: By analyzing the advantages and disadvantages of existing IIE models across the 16 dimensions, we offer valuable research insights to guide future development in the field. We will open-source I2EBench, including all instructions, input images, human annotations, edited images from all evaluated methods, and a simple script for evaluating the results from new IIE models. The code, dataset and generated images from all IIE models are provided in github: https://github.com/cocoshe/I2EBench.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
