arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.02779cs.CVcs.GRcs.LG
收藏

混合潜在扩散

Blended Latent Diffusion

The Hebrew University of Jerusalem(耶路撒冷希伯来大学);Reichman University(莱赫曼大学)

Omri Avrahami, Ohad Fried, Dani Lischinski

AI总结:

本文提出混合潜在扩散方法,将潜在扩散模型与混合扩散结合,实现快速且高精度的局部文本驱动图像编辑,并解决重建与细掩码问题。

摘要:

神经图像生成的巨大进步,加上看似全能的视觉-语言模型的出现,最终实现了基于文本的创建和编辑图像界面。处理通用图像需要多样化的底层生成模型,因此最新工作利用扩散模型,这些模型在多样性方面已证明超越GAN。然而,扩散模型的一个主要缺点是推理时间相对较慢。在本文中,我们提出了一种加速解决方案,用于通用图像的局部文本驱动编辑任务,其中所需编辑限于用户提供的掩码。我们的解决方案利用最近的文本到图像潜在扩散模型(LDM),该模型通过在较低维度的潜在空间中操作来加速扩散。我们首先通过将混合扩散融入LDM,将其转换为局部图像编辑器。接下来,我们提出一种基于优化的解决方案,以解决该LDM固有的无法准确重建图像的问题。最后,我们处理使用细掩码执行局部编辑的场景。我们定性和定量地评估了我们的方法,与现有基线相比,并证明除了更快之外,我们的方法在精度上优于基线,同时减轻了它们的一些伪影。

英文摘要:

The tremendous progress in neural image generation, coupled with the emergence of seemingly omnipotent vision-language models has finally enabled text-based interfaces for creating and editing images. Handling generic images requires a diverse underlying generative model, hence the latest works utilize diffusion models, which were shown to surpass GANs in terms of diversity. One major drawback of diffusion models, however, is their relatively slow inference time. In this paper, we present an accelerated solution to the task of local text-driven editing of generic images, where the desired edits are confined to a user-provided mask. Our solution leverages a recent text-to-image Latent Diffusion Model (LDM), which speeds up diffusion by operating in a lower-dimensional latent space. We first convert the LDM into a local image editor by incorporating Blended Diffusion into it. Next we propose an optimization-based solution for the inherent inability of this LDM to accurately reconstruct images. Finally, we address the scenario of performing local edits using thin masks. We evaluate our method against the available baselines both qualitatively and quantitatively and demonstrate that in addition to being faster, our method achieves better precision than the baselines while mitigating some of their artifacts.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑