基于扩散模型的单图像连续布局编辑
Continuous Layout Editing of Single Images with Diffusion Models
AI总结：
提出首个单图像布局编辑框架，通过掩码文本反演解耦对象概念和免训练优化实现布局控制，在保留视觉属性的同时支持连续编辑，效果优于现有基线。
摘要：
大规模文本到图像扩散模型的最新进展已经实现了图像编辑中的许多应用。然而，这些方法都无法编辑单个现有图像的布局。为了解决这一空白，我们提出了第一个在保留图像视觉属性的同时编辑单个图像布局的框架，从而允许对单个图像进行连续编辑。我们的方法通过两个关键模块实现。首先，为了保留图像中多个对象的特征，我们使用一种称为掩码文本反演的新方法，将不同对象的概念解耦，并将其嵌入到单独的文本标记中。接下来，我们提出了一种无需训练的优化方法，用于对预训练的扩散模型进行布局控制，这使我们能够使用学习到的概念重新生成图像，并将其与用户指定的布局对齐。作为第一个编辑现有图像布局的框架，我们证明了我们的方法是有效的，并且优于为支持此任务而修改的其他基线。我们的代码将在接受后免费向公众提供。
英文摘要：
Recent advancements in large-scale text-to-image diffusion models have enabled many applications in image editing. However, none of these methods have been able to edit the layout of single existing images. To address this gap, we propose the first framework for layout editing of a single image while preserving its visual properties, thus allowing for continuous editing on a single image. Our approach is achieved through two key modules. First, to preserve the characteristics of multiple objects within an image, we disentangle the concepts of different objects and embed them into separate textual tokens using a novel method called masked textual inversion. Next, we propose a training-free optimization method to perform layout control for a pre-trained diffusion model, which allows us to regenerate images with learned concepts and align them with user-specified layouts. As the first framework to edit the layout of existing images, we demonstrate that our method is effective and outperforms other baselines that were modified to support this task. Our code will be freely available for public use upon acceptance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
