arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2508.07607cs.CV

X2Edit:通过自构建数据与任务感知表示学习重新审视任意指令图像编辑

X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learning

OPPO AI Center(OPPO人工智能中心)

Jian Ma, Xujie Zhu, Zihao Pan, Qirong Peng, Xu Guo, Chen Chen, Haonan Lu

AI总结:

本文提出X2Edit数据集和基于FLUX.1的任务感知MoE-LoRA训练方法,通过自构建370万高质量数据和对比学习,实现了与社区生成模型兼容的高性能任意指令图像编辑。

摘要:

现有的面向任意指令图像编辑的开源数据集仍不够理想,同时缺乏一个与社区主流生成模型兼容的即插即用编辑模块。在本文中,我们首先引入了X2Edit数据集,这是一个涵盖14种不同编辑任务(包括主体驱动生成)的综合性数据集。我们利用业界领先的统一图像生成模型和专家模型来构建数据。同时,我们借助视觉语言模型(VLM)设计合理的编辑指令,并实现多种评分机制来筛选数据。最终,我们构建了370万个类别均衡的高质量数据。其次,为了更好地与社区图像生成模型无缝集成,我们基于FLUX.1设计了任务感知的MoE-LoRA训练,其参数量仅为完整模型的8%。为了进一步提升最终性能,我们利用扩散模型的内部表示,并根据图像编辑类型定义正样本和负样本,从而引入对比学习。大量实验表明,该模型的编辑性能在众多优秀模型中具有竞争力。此外,所构建的数据集相较于现有开源数据集展现出显著优势。X2Edit的开源代码、检查点和数据集可在以下链接获取:https://github.com/OPPO-Mente-Lab/X2Edit。

英文摘要:

Existing open-source datasets for arbitrary-instruction image editing remain suboptimal, while a plug-and-play editing module compatible with community-prevalent generative models is notably absent. In this paper, we first introduce the X2Edit Dataset, a comprehensive dataset covering 14 diverse editing tasks, including subject-driven generation. We utilize the industry-leading unified image generation models and expert models to construct the data. Meanwhile, we design reasonable editing instructions with the VLM and implement various scoring mechanisms to filter the data. As a result, we construct 3.7 million high-quality data with balanced categories. Second, to better integrate seamlessly with community image generation models, we design task-aware MoE-LoRA training based on FLUX.1, with only 8\% of the parameters of the full model. To further improve the final performance, we utilize the internal representations of the diffusion model and define positive/negative samples based on image editing types to introduce contrastive learning. Extensive experiments demonstrate that the model's editing performance is competitive among many excellent models. Additionally, the constructed dataset exhibits substantial advantages over existing open-source datasets. The open-source code, checkpoints, and datasets for X2Edit can be found at the following link: https://github.com/OPPO-Mente-Lab/X2Edit.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑