基于大语言模型的图像编辑评估：一个全面的基准和中间层探针方法
Evaluating Image Editing with LLMs: A Comprehensive Benchmark and Intermediate-Layer Probing Approach
AI总结：
本文提出TIEdit基准和EditProbe方法，通过5120张编辑图像和专家评分，评估文本引导图像编辑方法的感知质量、对齐性和内容保真度，同时利用大语言模型中间层探针提升评估准确性。
摘要：
评估文本引导图像编辑（TIE）方法仍是一个具有挑战性的问题，因为可靠的评估应同时考虑感知质量、与文本指令的一致性以及原始图像内容的保持。尽管TIE模型取得了快速进展，但现有评估基准在规模上仍然有限，且与人类感知判断的相关性较弱。在本文中，我们引入TIEdit，一个用于系统评估文本引导图像编辑方法的基准。TIEdit包含512张源图像与八个代表性编辑任务的编辑提示配对，生成5,120张由十个最先进的TIE模型生成的编辑图像。为了获得可靠的主观评分，招募了20名专家，产生307,200个原始主观评分，最终汇总为15,360个平均意见分数（MOSs），涵盖三个评估维度：感知质量、编辑对齐性和内容保持。除了基准本身外，我们还提出EditProbe，一种基于大语言模型的评估器，通过隐藏表示的中间层探针估计编辑质量。与仅依赖最终模型输出不同，EditProbe从多模态大语言模型的中间层提取信息性表示，以更好地捕捉源图像、编辑指令和编辑结果之间的语义和感知关系。实验结果表明，广泛使用的自动评估指标在编辑任务上与人类判断的相关性有限，而EditProbe在与人类感知的对齐方面实现了显著更强的关联。共同而言，TIEdit和EditProbe为文本引导图像编辑方法的更可靠和感知对齐的评估提供了基础。
英文摘要：
Evaluating text-guided image editing (TIE) methods remains a challenging problem, as reliable assessment should simultaneously consider perceptual quality, alignment with textual instructions, and preservation of original image content. Despite rapid progress in TIE models, existing evaluation benchmarks remain limited in scale and often show weak correlation with human perceptual judgments. In this work, we introduce TIEdit, a benchmark for systematic evaluation of text-guided image editing methods. TIEdit consists of 512 source images paired with editing prompts across eight representative editing tasks, producing 5,120 edited images generated by ten state-of-the-art TIE models. To obtain reliable subjective ratings, 20 experts are recruited to produce 307,200 raw subjective ratings, which accumulates into 15,360 mean opinion scores (MOSs) across three evaluation dimensions: perceptual quality, editing alignment, and content preservation. Beyond the benchmark itself, we further propose EditProbe, an LLM-based evaluator that estimates editing quality via intermediate-layer probing of hidden representations. Instead of relying solely on final model outputs, EditProbe extracts informative representations from intermediate layers of multimodal large language models to better capture semantic and perceptual relationships between source images, editing instructions, and edited results. Experimental results demonstrate that widely used automatic evaluation metrics show limited correlation with human judgments on editing tasks, while EditProbe achieves substantially stronger alignment with human perception. Together, TIEdit and EditProbe provide a foundation for more reliable and perceptually aligned evaluation of text-guided image editing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
