arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.03963cs.CV
收藏

InfinityGAN:无限像素图像合成

InfinityGAN: Towards Infinite-Pixel Image Synthesis

UC Merced(加州大学默塞德分校);Snap Inc.(色拉布公司);Carnegie Mellon University(卡内基梅隆大学);Yonsei University(延世大学);Google Research(谷歌研究院)

Chieh Hubert Lin, Hsin-Ying Lee, Yen-Chi Cheng, Sergey Tulyakov, Ming-Hsuan Yang

摘要:

我们提出了一个新的框架,InfinityGAN,任意大小的图像生成。这项任务涉及若干关键挑战。首先,在计算和大视场训练数据的可用性方面,将现有模型缩放到任意大的图像尺寸是资源受限的。InfinityGAN以无缝的逐补丁方式进行训练和推断,计算资源少。其次,大图像应该在局部和全局上保持一致,避免重复模式,并且看起来逼真。为了解决这些问题,InfinityGAN解开了全局外观,局部结构和纹理。有了这个公式,我们可以生成以前无法达到的空间大小和细节水平的图像。实验评估验证了InfinityGAN生成的图像与基线相比具有更高的真实性,并具有并行推理功能。最后,我们展示了我们的方法解锁的几个应用程序,如空间风格融合,多模式outpainting和图像之间。所有应用程序都可以使用任意输入和输出大小进行操作。请在https://openreview.net/forum?上找到该论文的完整版本id=ufGMqIM0a4b。

英文摘要:

We present a novel framework, InfinityGAN, for arbitrary-sized image generation. The task is associated with several key challenges. First, scaling existing models to an arbitrarily large image size is resource-constrained, in terms of both computation and availability of large-field-of-view training data. InfinityGAN trains and infers in a seamless patch-by-patch manner with low computational resources. Second, large images should be locally and globally consistent, avoid repetitive patterns, and look realistic. To address these, InfinityGAN disentangles global appearances, local structures, and textures. With this formulation, we can generate images with spatial size and level of details not attainable before. Experimental evaluation validates that InfinityGAN generates images with superior realism compared to baselines and features parallelizable inference. Finally, we show several applications unlocked by our approach, such as spatial style fusion, multi-modal outpainting, and image inbetweening. All applications can be operated with arbitrary input and output sizes. Please find the full version of the paper at https://openreview.net/forum?id=ufGMqIM0a4b .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑