arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.15738cs.CV

AnyEdit:掌握任意想法的统一高质量图像编辑

AnyEdit: Mastering Unified High-Quality Image Editing for Any Idea

Zhejiang University(浙江大学);Nanyang Technological University(南洋理工大学);Ant Group(蚂蚁集团)

Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang, Yueting Zhuang

AI总结:

本文提出AnyEdit数据集及基于扩散的统一图像编辑模型,通过任务感知路由和可学习任务嵌入,在三个基准上显著提升编辑性能。

摘要:

基于指令的图像编辑旨在通过自然语言指令修改特定的图像元素。然而,该领域的现有模型往往难以准确执行复杂的用户指令,因为它们是在编辑类型有限的低质量数据上训练的。我们提出了AnyEdit,一个全面的多模态指令编辑数据集,包含超过20种编辑类型和五个领域的250万高质量编辑对。我们通过三个方面确保AnyEdit集合的多样性和质量:初始数据多样性、自适应编辑过程和编辑结果的自动选择。利用该数据集,我们进一步训练了一种新颖的AnyEdit Stable Diffusion模型,该模型具有任务感知路由和可学习的任务嵌入,用于统一图像编辑。在三个基准数据集上的综合实验表明,AnyEdit持续提升了基于扩散的编辑模型的性能。这为开发支持人类创造力的指令驱动图像编辑模型提供了前景。

英文摘要:

Instruction-based image editing aims to modify specific image elements with natural language instructions. However, current models in this domain often struggle to accurately execute complex user instructions, as they are trained on low-quality data with limited editing types. We present AnyEdit, a comprehensive multi-modal instruction editing dataset, comprising 2.5 million high-quality editing pairs spanning over 20 editing types and five domains. We ensure the diversity and quality of the AnyEdit collection through three aspects: initial data diversity, adaptive editing process, and automated selection of editing results. Using the dataset, we further train a novel AnyEdit Stable Diffusion with task-aware routing and learnable task embedding for unified image editing. Comprehensive experiments on three benchmark datasets show that AnyEdit consistently boosts the performance of diffusion-based editing models. This presents prospects for developing instruction-driven image editing models that support human creativity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑