发生了什么变化？利用多模态大语言模型检测和评估指令引导的图像编辑
What Changed? Detecting and Evaluating Instruction-Guided Image Edits with Multimodal Large Language Models
AI总结：
针对指令引导图像编辑评估难的问题，提出基于多模态大语言模型的DICE模型，通过差异检测与一致性估计两阶段方法，实现与人类判断高度相关的编辑质量评估。
摘要：
基于指令的图像编辑模型在生成任务中提供了更多的个性化机会。然而，正确评估其结果具有挑战性，且大多数现有指标在与人类判断的一致性和可解释性方面存在不足。为解决这些问题，我们引入了DICE（差异一致性估计器），一个旨在检测原始图像与编辑后图像之间的局部差异并评估其与给定修改请求相关性的模型。DICE由两个关键组件组成：差异检测器和一致性估计器，两者均基于自回归多模态大语言模型（MLLM）构建，并采用一种利用自监督、从修复网络蒸馏以及全监督的训练策略进行训练。通过大量实验，我们评估了管道的每个阶段，在提出的框架内比较了不同的多模态大语言模型。我们证明，DICE能够有效识别一致的编辑，并能以与人类判断高度相关的方式有效评估不同编辑模型生成的图像。我们公开发布了源代码、模型和数据。
英文摘要：
Instruction-based image editing models offer increased personalization opportunities in generative tasks. However, properly evaluating their results is challenging, and most of the existing metrics lag in terms of alignment with human judgment and explainability. To tackle these issues, we introduce DICE (DIfference Coherence Estimator), a model designed to detect localized differences between the original and the edited image and to assess their relevance to the given modification request. DICE consists of two key components: a difference detector and a coherence estimator, both built on an autoregressive Multimodal Large Language Model (MLLM) and trained using a strategy that leverages self-supervision, distillation from inpainting networks, and full supervision. Through extensive experiments, we evaluate each stage of our pipeline, comparing different MLLMs within the proposed framework. We demonstrate that DICE effectively identifies coherent edits, effectively evaluating images generated by different editing models with a strong correlation with human judgment. We publicly release our source code, models, and data.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
