arXiv 2507.161932025-09-07cs.CVcs.MM

LMM4Edit：使用LMM评估与基准测试多模态图像编辑

LMM4Edit: Benchmarking and Evaluating Multimodal Image Editing with LMMs

Institute of Image Communication and Network Engineering, Shanghai JiaoTong University（上海交通大学图像通信与网络工程研究所）；University of Electronic and Science Technology of China（电子科技大学）；Nanyang Technological University（南洋理工大学）

Zitong Xu, Huiyu Duan, Bingnan Liu, Guangji Ma, Jiarui Wang, Liu Yang, Shiqi Gao, Xiaoyu Wang, Jia Wang, Xiongkuo Min, Guangtao Zhai, Weisi Lin

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对文本引导图像编辑评估的不足，提出EBench-18K大规模基准和LMM4Edit指标，基于LMM从多维度评估编辑质量，实验证明其与人类偏好高度一致且具有泛化能力。

摘要：

文本引导图像编辑（TIE）的快速发展使得通过文本提示进行图像修改成为可能。然而，当前的TIE模型在平衡图像质量、编辑对齐度以及与原始图像的一致性方面仍面临挑战，限制了其实际应用。现有的TIE评估基准和指标在规模或与人类感知的一致性方面存在局限性。为此，我们引入了EBench-18K，这是首个大规模图像编辑基准，包含18K张带有细粒度人类偏好标注的编辑图像，用于评估TIE。具体而言，EBench-18K包含1,080张源图像及其对应的跨21个任务的编辑提示，由17个最先进的TIE模型生成的18K+张编辑图像，从三个评估维度评定的55K+个平均意见分数（MOS），以及18K+个问答（QA）对。基于EBench-18K，我们使用杰出的LMM评估编辑图像，而评估结果反过来为评估LMM的理解能力与人类偏好之间的一致性提供了见解。然后，我们提出了LMM4Edit，一种基于LMM的指标，以一体化方式从感知质量、编辑对齐度、属性保留和任务特定QA准确性方面评估图像编辑模型。大量实验表明，LMM4Edit取得了出色的性能，并与人类偏好高度一致。在其他数据集上的零样本验证也显示了我们的模型的泛化能力。数据集和代码可在https://github.com/IntMeGroup/LMM4Edit获取。

英文摘要：

The rapid advancement of Text-guided Image Editing (TIE) enables image modifications through text prompts. However, current TIE models still struggle to balance image quality, editing alignment, and consistency with the original image, limiting their practical applications. Existing TIE evaluation benchmarks and metrics have limitations on scale or alignment with human perception. To this end, we introduce EBench-18K, the first large-scale image Editing Benchmark including 18K edited images with fine-grained human preference annotations for evaluating TIE. Specifically, EBench-18K includes 1,080 source images with corresponding editing prompts across 21 tasks, 18K+ edited images produced by 17 state-of-the-art TIE models, 55K+ mean opinion scores (MOSs) assessed from three evaluation dimensions, and 18K+ question-answering (QA) pairs. Based on EBench-18K, we employ outstanding LMMs to assess edited images, while the evaluation results, in turn, provide insights into assessing the alignment between the LMMs' understanding ability and human preferences. Then, we propose LMM4Edit, a LMM-based metric for evaluating image Editing models from perceptual quality, editing alignment, attribute preservation, and task-specific QA accuracy in an all-in-one manner. Extensive experiments show that LMM4Edit achieves outstanding performance and aligns well with human preference. Zero-shot validation on the other datasets also shows the generalization ability of our model. The dataset and code are available at https://github.com/IntMeGroup/LMM4Edit.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。