arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2209.10948cs.CVcs.LG
收藏

文本到图像生成的扩散模型的实现和实验

Implementing and Experimenting with Diffusion Models for Text-to-Image Generation

École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院)

Robin Zbinden

摘要:

利用深度学习的许多最新进展,文本到图像生成模型目前具有吸引公众注意力的优点。其中两个模型,DALL-E 2和Imagen,已经证明了高度逼真的图像可以从一个简单的文本描述的图像。基于一种称为扩散模型的图像生成新方法,文本到图像模型使得能够生成许多不同类型的高分辨率图像,其中人类的想象力是唯一的限制。然而,这些模型需要非常大量的计算资源来进行训练,以及处理从互联网上收集的庞大数据集。此外,代码库和模型都没有发布。因此,它阻止了人工智能社区对这些尖端模型的试验,使其结果的复制变得复杂,如果不是不可能的话。在这篇论文中,我们的目标是通过首先回顾这些模型所使用的不同方法和技术,然后提出我们自己的文本到图像模型的实现。高度基于DALL-E 2,我们引入了几个轻微的修改,以解决高计算成本。因此,我们有机会进行实验,以了解这些模型的能力,特别是在低资源制度。特别是,我们提供了比DALL-E 2作者进行的分析更深入的额外分析,包括消融研究。此外,扩散模型使用所谓的指导方法来帮助生成过程。我们介绍了一种新的指导方法,可与其他指导方法,以提高图像质量。最后,我们的模型生成的图像质量相当好,而不必承受最先进的文本到图像模型的大量训练成本。

英文摘要:

Taking advantage of the many recent advances in deep learning, text-to-image generative models currently have the merit of attracting the general public attention. Two of these models, DALL-E 2 and Imagen, have demonstrated that highly photorealistic images could be generated from a simple textual description of an image. Based on a novel approach for image generation called diffusion models, text-to-image models enable the production of many different types of high resolution images, where human imagination is the only limit. However, these models require exceptionally large amounts of computational resources to train, as well as handling huge datasets collected from the internet. In addition, neither the codebase nor the models have been released. It consequently prevents the AI community from experimenting with these cutting-edge models, making the reproduction of their results complicated, if not impossible. In this thesis, we aim to contribute by firstly reviewing the different approaches and techniques used by these models, and then by proposing our own implementation of a text-to-image model. Highly based on DALL-E 2, we introduce several slight modifications to tackle the high computational cost induced. We thus have the opportunity to experiment in order to understand what these models are capable of, especially in a low resource regime. In particular, we provide additional and analyses deeper than the ones performed by the authors of DALL-E 2, including ablation studies. Besides, diffusion models use so-called guidance methods to help the generating process. We introduce a new guidance method which can be used in conjunction with other guidance methods to improve image quality. Finally, the images generated by our model are of reasonably good quality, without having to sustain the significant training costs of state-of-the-art text-to-image models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑