InsightEdit：提升图像编辑的指令遵循能力
InsightEdit: Towards Better Instruction Following for Image Editing
AI总结：
本文针对指令图像编辑任务，提出AdvancedEdit数据集解决现有数据缺陷，并引入双流桥接机制利用MLLM的文本与视觉特征，实现更精准的编辑，提升复杂指令遵循和背景一致性。
摘要：
本文聚焦于基于指令的图像编辑任务。此前的工作如InstructPix2Pix、InstructDiffusion和SmartEdit已探索端到端编辑，但仍存在两个局限：一是现有数据集分辨率低、背景一致性差且指令过于简单；二是当前方法主要以文本为条件，未充分利用丰富的图像信息，导致在复杂指令遵循和背景一致性维持方面表现不佳。针对这些问题，我们首先通过新型数据构建 pipeline 构建了AdvancedEdit数据集，形成具有高视觉质量、复杂指令和良好背景一致性的大规模数据集。然后，为进一步注入丰富图像信息，我们引入双流桥接机制，利用多模态大型语言模型（MLLM）推理的文本和视觉特征，更精准地引导图像编辑过程。
英文摘要：
In this paper, we focus on the task of instruction-based image editing. Previous works like InstructPix2Pix, InstructDiffusion, and SmartEdit have explored end-to-end editing. However, two limitations still remain: First, existing datasets suffer from low resolution, poor background consistency, and overly simplistic instructions. Second, current approaches mainly condition on the text while the rich image information is underexplored, therefore inferior in complex instruction following and maintaining background consistency. Targeting these issues, we first curated the AdvancedEdit dataset using a novel data construction pipeline, formulating a large-scale dataset with high visual quality, complex instructions, and good background consistency. Then, to further inject the rich image information, we introduce a two-stream bridging mechanism utilizing both the textual and visual features reasoned by the powerful Multimodal Large Language Models (MLLM) to guide the image editing process more precisely. Extensive results demonstrate that our approach, InsightEdit, achieves state-of-the-art performance, excelling in complex instruction following and maintaining high background consistency with the original image.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
