SuperEdit：矫正并促进基于指令的图像编辑的监督信号
SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based Image Editing
AI总结：
本文提出SuperEdit，通过矫正编辑指令并利用正负指令构建对比监督信号，在不依赖VLM模块或预训练任务的情况下，以更少数据和更小模型显著提升基于指令的图像编辑性能。
摘要：
由于人工收集精确编辑数据存在困难，现有数据集通常采用各种自动化方法构建，导致编辑指令与原始-编辑图像对之间不匹配，从而产生噪声监督信号。近期工作尝试通过生成更高质量的编辑图像、在识别任务上进行预训练或引入视觉-语言模型（VLMs）来改进编辑模型，但未能解决这一根本问题。本文提出一种新颖的解决方案：为给定的图像对构建更有效的编辑指令。具体包括矫正编辑指令，使其更好地与原始-编辑图像对对齐，并使用对比编辑指令进一步增强其有效性。具体而言，我们发现编辑模型在不同推理步骤中表现出特定的生成属性，且这些属性与文本无关。基于这些先验属性，我们为VLMs定义了一个统一的引导准则来矫正编辑指令。然而，有些具有挑战性的编辑场景仅靠矫正后的指令无法解决。为此，我们进一步利用正指令和负指令构建对比监督信号，并通过三元组损失将其引入模型训练，从而进一步促进监督有效性。我们的方法不需要先前工作所用的VLM模块或预训练任务，提供了一种更直接、高效的方式来提供更好的监督信号，并为基于指令的图像编辑提供了一种新颖、简单且有效的解决方案。在多个基准上的结果表明，我们的方法显著优于现有方法。与先前最优的SmartEdit相比，我们在Real-Edit基准上以少30倍的训练数据和13倍小的模型规模取得了9.19%的提升。
英文摘要：
Due to the challenges of manually collecting accurate editing data, existing datasets are typically constructed using various automated methods, leading to noisy supervision signals caused by the mismatch between editing instructions and original-edited image pairs. Recent efforts attempt to improve editing models through generating higher-quality edited images, pre-training on recognition tasks, or introducing vision-language models (VLMs) but fail to resolve this fundamental issue. In this paper, we offer a novel solution by constructing more effective editing instructions for given image pairs. This includes rectifying the editing instructions to better align with the original-edited image pairs and using contrastive editing instructions to further enhance their effectiveness. Specifically, we find that editing models exhibit specific generation attributes at different inference steps, independent of the text. Based on these prior attributes, we define a unified guide for VLMs to rectify editing instructions. However, there are some challenging editing scenarios that cannot be resolved solely with rectified instructions. To this end, we further construct contrastive supervision signals with positive and negative instructions and introduce them into the model training using triplet loss, thereby further facilitating supervision effectiveness. Our method does not require the VLM modules or pre-training tasks used in previous work, offering a more direct and efficient way to provide better supervision signals, and providing a novel, simple, and effective solution for instruction-based image editing. Results on multiple benchmarks demonstrate that our method significantly outperforms existing approaches. Compared with previous SOTA SmartEdit, we achieve 9.19% improvements on the Real-Edit benchmark with 30x less training data and 13x smaller model size.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
