arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.04820cs.CV
收藏

二元潜在扩散

Binary Latent Diffusion

Purdue University(普渡大学);Microsoft Corporation(微软公司)

Ze Wang, Jiang Wang, Zicheng Liu, Qiang Qiu

AI总结:

本文提出二元潜在扩散方法,通过伯努利自编码器将图像表示为二元潜在向量,并用二元潜在扩散模型建模其先验,在保持与最先进方法相当生成质量的同时,将采样效率提升至16步,并支持1024×1024高分辨率生成。

摘要:

在本文中,我们展示了二元潜在空间可用于探索紧凑而富有表现力的图像表示。我们通过训练一个具有伯努利编码分布的自编码器来建模图像与相应潜在二元表示之间的双向映射。一方面,二元潜在空间提供了一种紧凑的离散图像表示,其分布可以比像素或连续潜在表示更高效地建模。另一方面,我们现在将每个图像块表示为一个二元向量,而不是像基于向量量化的离散图像表示那样使用学习码本的索引。通过这种方式,我们获得了二元潜在表示,能够在潜在空间中无需任何多阶段层次结构即可实现更好的图像质量和更高分辨率的图像表示。在这个二元潜在空间中,现在可以使用专门为建模二元图像表示先验而设计的二元潜在扩散模型有效地生成图像。我们在多个数据集上进行了条件生成和无条件生成实验,并表明所提出的方法在性能上与最先进方法相当,同时将采样效率大幅提高到仅需16步,且无需使用任何测试时加速。所提出的框架还可以无缝扩展到$1024 \ imes 1024$高分辨率图像生成,而无需借助潜在层次结构或多阶段细化。

英文摘要:

In this paper, we show that a binary latent space can be explored for compact yet expressive image representations. We model the bi-directional mappings between an image and the corresponding latent binary representation by training an auto-encoder with a Bernoulli encoding distribution. On the one hand, the binary latent space provides a compact discrete image representation of which the distribution can be modeled more efficiently than pixels or continuous latent representations. On the other hand, we now represent each image patch as a binary vector instead of an index of a learned cookbook as in discrete image representations with vector quantization. In this way, we obtain binary latent representations that allow for better image quality and high-resolution image representations without any multi-stage hierarchy in the latent space. In this binary latent space, images can now be generated effectively using a binary latent diffusion model tailored specifically for modeling the prior over the binary image representations. We present both conditional and unconditional image generation experiments with multiple datasets, and show that the proposed method performs comparably to state-of-the-art methods while dramatically improving the sampling efficiency to as few as 16 steps without using any test-time acceleration. The proposed framework can also be seamlessly scaled to $1024 \times 1024$ high-resolution image generation without resorting to latent hierarchy or multi-stage refinements.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑