arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.11273cs.CVcs.AI
收藏

GR-GAN:逐步细化文本到图像生成

GR-GAN: Gradual Refinement Text-to-image Generation

Beijing University of Posts and Telecommunications(北京邮电大学)

Bo Yang, Fangxiang Feng, Xiaojie Wang

摘要:

一个好的文本到图像模型不仅要生成高质量的图像,而且要保证文本和生成的图像之间的一致性。以前的模型未能同时很好地修复两侧。本文提出了一种渐进精化生成对抗网络(GR-GAN)来有效地解决该问题。GRG模块用于生成从低分辨率到高分辨率的图像,对应的文本约束从粗粒度(句子)到细粒度(单词)逐段生成,ITM模块用于提供相应阶段的文本-图像和单词-区域两级的图像-文本匹配损失。我们还引入了一个新的度量跨模型距离(CMD),同时评估图像质量和图像-文本的一致性。实验结果表明,GR-GAN显著优于以往的模型,并实现了新的国家的最先进的FID和CMD。详细的分析表明,在GR-GAN中的不同生成阶段的效率。

英文摘要:

A good Text-to-Image model should not only generate high quality images, but also ensure the consistency between the text and the generated image. Previous models failed to simultaneously fix both sides well. This paper proposes a Gradual Refinement Generative Adversarial Network (GR-GAN) to alleviates the problem efficiently. A GRG module is designed to generate images from low resolution to high resolution with the corresponding text constraints from coarse granularity (sentence) to fine granularity (word) stage by stage, a ITM module is designed to provide image-text matching losses at both sentence-image level and word-region level for corresponding stages. We also introduce a new metric Cross-Model Distance (CMD) for simultaneously evaluating image quality and image-text consistency. Experimental results show GR-GAN significant outperform previous models, and achieve new state-of-the-art on both FID and CMD. A detailed analysis demonstrates the efficiency of different generation stages in GR-GAN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑