OmniEdit：通过专家监督构建图像编辑通用模型
OmniEdit: Building Image Editing Generalist Models Through Specialist Supervision
AI总结：
本文提出全能图像编辑器OmniEdit，通过七个专家模型监督、GPT-4o重要性采样、EditNet架构及多长宽比数据训练，解决现有方法技能受限、数据噪声大和分辨率固定问题，显著优于现有模型。
摘要：
指令引导的图像编辑方法通过在自动合成或手动标注的图像编辑对上训练扩散模型，展现出了巨大的潜力。然而，这些方法距离实际的现实生活应用仍然相差甚远。我们指出了导致这一差距的三个主要挑战。首先，由于合成过程存在偏差，现有模型的编辑技能有限。其次，这些方法使用包含大量噪声和伪影的数据集进行训练。这是因为应用了如 CLIP-score 等简单的过滤方法。第三，所有这些数据集都局限于单一的低分辨率和固定的长宽比，限制了对真实世界用例的通用性。在本文中，我们提出了 OmniEdit，这是一个全能的编辑器，能够无缝处理七种不同的图像编辑任务，并支持任意长宽比。我们的贡献有四个方面：(1) OmniEdit 通过利用七个不同专家模型的监督进行训练，以确保任务覆盖率。(2) 我们利用基于大型多模态模型（如 GPT-4o）提供的分数的重要性采样，而不是 CLIP-score，来提高数据质量。(3) 我们提出了一种名为 EditNet 的新编辑架构，以大幅提高编辑成功率。(4) 我们提供了具有不同长宽比的图像，以确保我们的模型能够处理野外的任何图像。我们精心整理了一个测试集，其中包含不同长宽比的图像，并伴有多样化的指令以涵盖不同的任务。自动评估和人工评估均表明，OmniEdit 能够显著优于所有现有模型。我们的代码、数据集和模型将在 https://tiger-ai-lab.github.io/OmniEdit/ 上提供。
英文摘要：
Instruction-guided image editing methods have demonstrated significant potential by training diffusion models on automatically synthesized or manually annotated image editing pairs. However, these methods remain far from practical, real-life applications. We identify three primary challenges contributing to this gap. Firstly, existing models have limited editing skills due to the biased synthesis process. Secondly, these methods are trained with datasets with a high volume of noise and artifacts. This is due to the application of simple filtering methods like CLIP-score. Thirdly, all these datasets are restricted to a single low resolution and fixed aspect ratio, limiting the versatility to handle real-world use cases. In this paper, we present \omniedit, which is an omnipotent editor to handle seven different image editing tasks with any aspect ratio seamlessly. Our contribution is in four folds: (1) \omniedit is trained by utilizing the supervision from seven different specialist models to ensure task coverage. (2) we utilize importance sampling based on the scores provided by large multimodal models (like GPT-4o) instead of CLIP-score to improve the data quality. (3) we propose a new editing architecture called EditNet to greatly boost the editing success rate, (4) we provide images with different aspect ratios to ensure that our model can handle any image in the wild. We have curated a test set containing images of different aspect ratios, accompanied by diverse instructions to cover different tasks. Both automatic evaluation and human evaluations demonstrate that \omniedit can significantly outperform all the existing models. Our code, dataset and model will be available at https://tiger-ai-lab.github.io/OmniEdit/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
