arXiv 2509.146382025-09-18cs.CV

MultiEdit：在多样且具有挑战性的任务上推进基于指令的图像编辑

MultiEdit: Advancing Instruction-based Image Editing on Diverse and Challenging Tasks

University of New South Wales（新南威尔士大学）；The University of Hong Kong（香港大学）；Zhejiang University（浙江大学）；Westlake University（西湖大学）

Mingsong Li, Lin Liu, Hongjun Wang, Haoxing Chen, Xijun Gu, Shizhan Liu, Dong Gong, Junbo Zhao, Zhenzhong Lan, Jianguo Li

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MultiEdit构建了超107K样本的综合图像编辑数据集，覆盖6类挑战性任务，并利用双MLLM流程生成高质量编辑数据，微调后显著提升开源模型在复杂编辑任务上的表现。

摘要：

当前基于指令的图像编辑（IBIE）方法在具有挑战性的编辑任务上表现不佳，因为现有数据集的编辑类型和样本数量都有限。此外，传统数据集构建通常包含有噪声的图像-文本对，这可能引入偏差并限制模型在复杂编辑场景中的能力。为解决这些局限，我们提出了MultiEdit，一个包含超过107K高质量图像编辑样本的综合数据集。它通过18种非风格迁移编辑类型和38种风格迁移操作的多样集合涵盖6项具有挑战性的编辑任务，覆盖从复杂风格迁移到复杂语义操作（如人物参考编辑和图像内文本编辑）的广泛范围。我们采用一种新颖的数据集构建流程，利用两个多模态大语言模型（MLLMs）分别生成视觉自适应的编辑指令和生成高保真编辑图像。大量实验表明，使用我们的MultiEdit-Train集对基础开源模型进行微调，可显著提升模型在我们提出的MultiEdit-Test基准上处理复杂编辑任务的性能，同时有效保持其在标准编辑基准上的能力。我们相信MultiEdit为推进更多样且更具挑战性的IBIE能力研究提供了宝贵资源。我们的数据集可在 https://huggingface.co/datasets/inclusionAI/MultiEdit 获取。

英文摘要：

Current instruction-based image editing (IBIE) methods struggle with challenging editing tasks, as both editing types and sample counts of existing datasets are limited. Moreover, traditional dataset construction often contains noisy image-caption pairs, which may introduce biases and limit model capabilities in complex editing scenarios. To address these limitations, we introduce MultiEdit, a comprehensive dataset featuring over 107K high-quality image editing samples. It encompasses 6 challenging editing tasks through a diverse collection of 18 non-style-transfer editing types and 38 style transfer operations, covering a spectrum from sophisticated style transfer to complex semantic operations like person reference editing and in-image text editing. We employ a novel dataset construction pipeline that utilizes two multi-modal large language models (MLLMs) to generate visual-adaptive editing instructions and produce high-fidelity edited images, respectively. Extensive experiments demonstrate that fine-tuning foundational open-source models with our MultiEdit-Train set substantially improves models' performance on sophisticated editing tasks in our proposed MultiEdit-Test benchmark, while effectively preserving their capabilities on the standard editing benchmark. We believe MultiEdit provides a valuable resource for advancing research into more diverse and challenging IBIE capabilities. Our dataset is available at https://huggingface.co/datasets/inclusionAI/MultiEdit.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。