arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.01952cs.CVcs.AI
收藏

SDXL:改进潜在扩散模型以实现高分辨率图像合成

SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis

Stability AI(稳定性人工智能公司)

Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, Robin Rombach

AI总结:

SDXL通过三倍规模UNet骨干、双文本编码器、多宽高比训练和细化模型,大幅提升Stable Diffusion的高分辨率文本到图像合成性能,达到与黑盒最先进图像生成器竞争的水平。

摘要:

我们提出了SDXL,一种用于文本到图像合成的潜在扩散模型。与之前版本的Stable Diffusion相比,SDXL采用了三倍规模的UNet骨干网络:模型参数的增加主要源于更多的注意力模块和更大的交叉注意力上下文,因为SDXL使用了第二个文本编码器。我们设计了多种新颖的条件化方案,并在多种宽高比上训练SDXL。我们还引入了一个细化模型,该模型通过事后图像到图像技术用于提升SDXL生成样本的视觉保真度。我们证明,SDXL相比之前版本的Stable Diffusion表现出大幅提升的性能,并取得了与黑盒最先进图像生成器相竞争的结果。本着促进开放研究以及提升大模型训练与评估透明度的精神,我们在https://github.com/Stability-AI/generative-models提供代码和模型权重的访问。

英文摘要:

We present SDXL, a latent diffusion model for text-to-image synthesis. Compared to previous versions of Stable Diffusion, SDXL leverages a three times larger UNet backbone: The increase of model parameters is mainly due to more attention blocks and a larger cross-attention context as SDXL uses a second text encoder. We design multiple novel conditioning schemes and train SDXL on multiple aspect ratios. We also introduce a refinement model which is used to improve the visual fidelity of samples generated by SDXL using a post-hoc image-to-image technique. We demonstrate that SDXL shows drastically improved performance compared the previous versions of Stable Diffusion and achieves results competitive with those of black-box state-of-the-art image generators. In the spirit of promoting open research and fostering transparency in large model training and evaluation, we provide access to code and model weights at https://github.com/Stability-AI/generative-models

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑