arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.11058cs.CV
收藏

基于扩散模型的表示学习

Representation Learning with Diffusion Models

Heidelberg University(海德堡大学);Ludwig Maximilian University of Munich(慕尼黑大学)

Jeremias Traub

摘要:

扩散模型(DM)在图像合成任务和密度估计方面取得了最新的成果。应用于功能强大的预训练自动编码器(LDM)的潜在空间中,可以在不牺牲采样质量的情况下显着降低其巨大的计算需求。然而,DM和LDM缺乏语义上有意义的表示空间,因为扩散过程逐渐破坏潜变量中的信息。我们介绍了一个框架,学习这样的表示与扩散模型(LRDM)。为此,LDM以通过单独的编码器从干净图像中提取的表示为条件。特别地,DM和表示编码器被联合训练,以便学习特定于生成去噪过程的丰富表示。通过引入一个易于处理的表示先验,我们可以有效地从表示分布中进行无条件图像合成,而无需训练任何额外的模型。我们证明,i)有竞争力的图像生成结果可以实现与图像参数化的LDM,ii)LRDM能够学习语义有意义的表示,允许忠实的图像重建和语义插值。我们的实现可以在https://github.com/jeremiastraub/diffusion上获得。

英文摘要:

Diffusion models (DMs) have achieved state-of-the-art results for image synthesis tasks as well as density estimation. Applied in the latent space of a powerful pretrained autoencoder (LDM), their immense computational requirements can be significantly reduced without sacrificing sampling quality. However, DMs and LDMs lack a semantically meaningful representation space as the diffusion process gradually destroys information in the latent variables. We introduce a framework for learning such representations with diffusion models (LRDM). To that end, a LDM is conditioned on the representation extracted from the clean image by a separate encoder. In particular, the DM and the representation encoder are trained jointly in order to learn rich representations specific to the generative denoising process. By introducing a tractable representation prior, we can efficiently sample from the representation distribution for unconditional image synthesis without training of any additional model. We demonstrate that i) competitive image generation results can be achieved with image-parameterized LDMs, ii) LRDMs are capable of learning semantically meaningful representations, allowing for faithful image reconstructions and semantic interpolations. Our implementation is available at https://github.com/jeremiastraub/diffusion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑