DreamEdit：主题驱动的图像编辑
DreamEdit: Subject-driven Image Editing
AI总结：
本文提出主题替换与主题添加两个主题驱动图像编辑子任务，构建 DreamEditBench 数据集，并通过迭代生成的 DreamEditor 评估现有方法的局限。
摘要：
主题驱动图像生成旨在生成包含定制主题的图像，近来受到研究界的广泛关注。然而，以往工作无法精确控制目标主题的背景和位置。在本研究中，我们希望填补这一空白，并提出两个新颖的主题驱动子任务，即主题替换和主题添加。新任务在多个方面具有挑战性：用定制主题替换某一主题可能改变其形状、纹理和颜色，而将目标主题添加到给定场景中的指定位置，则需要具备情境感知的姿态。为攻克这两个新任务，我们首先人工整理了一个新数据集 DreamEditBench，其中包含22种不同类型的主题，以及440张具有不同难度级别的源图像。我们计划将 DreamEditBench 作为一个平台托管，并聘请训练有素的评估人员进行标准化人工评估。我们还设计了一种创新方法 DreamEditor，通过执行迭代生成来解决这些任务，从而能够平滑适配定制主题。在本项目中，我们开展自动评估和人工评估，以了解 DreamEditor 与基线方法在 DreamEditBench 上的性能。对于主题替换，我们发现现有模型对原始主题的形状和颜色很敏感；当源主题与目标主题差异很大时，模型失败率会急剧上升。对于主题添加，我们发现现有模型难以将定制主题平滑融入背景，导致生成图像出现明显伪影。我们希望 DreamEditBench 能成为标准平台，推动未来研究构建更可控的主题驱动图像编辑。我们的项目主页是 https://dreameditbenchteam.github.io/。
英文摘要：
Subject-driven image generation aims at generating images containing customized subjects, which has recently drawn enormous attention from the research community. However, the previous works cannot precisely control the background and position of the target subject. In this work, we aspire to fill the void and propose two novel subject-driven sub-tasks, i.e., Subject Replacement and Subject Addition. The new tasks are challenging in multiple aspects: replacing a subject with a customized one can change its shape, texture, and color, while adding a target subject to a designated position in a provided scene necessitates a context-aware posture. To conquer these two novel tasks, we first manually curate a new dataset DreamEditBench containing 22 different types of subjects, and 440 source images with different difficulty levels. We plan to host DreamEditBench as a platform and hire trained evaluators for standard human evaluation. We also devise an innovative method DreamEditor to resolve these tasks by performing iterative generation, which enables a smooth adaptation to the customized subject. In this project, we conduct automatic and human evaluations to understand the performance of DreamEditor and baselines on DreamEditBench. For Subject Replacement, we found that the existing models are sensitive to the shape and color of the original subject. The model failure rate will dramatically increase when the source and target subjects are highly different. For Subject Addition, we found that the existing models cannot easily blend the customized subjects into the background smoothly, leading to noticeable artifacts in the generated image. We hope DreamEditBench can become a standard platform to enable future investigations toward building more controllable subject-driven image editing. Our project homepage is https://dreameditbenchteam.github.io/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
