基于U-Transformer的广义图像外绘
Generalised Image Outpainting with U-Transformer
摘要：
在本文中，我们开发了一种新的基于变换的生成对抗神经网络称为U-变压器的广义图像outpainting问题。与目前大多数进行水平外推的图像外绘方法不同，我们的广义图像外绘可以外推视觉上下文周围的一个给定的图像与合理的结构和细节，即使是复杂的风景，建筑物和艺术图像。具体来说，我们设计了一个发电机作为一个编码器到解码器的结构嵌入流行的Swin Transformer块。因此，我们的新型神经网络可以更好地处理图像的长程依赖关系，这对于广义图像的外绘至关重要。另外，我们提出了一个U形结构和多视图的时间空间预测（TSP）模块，以加强图像的自我重建以及未知部分的预测顺利和逼真。在测试阶段，通过调整TSP模块中的预测步长，我们可以生成给定输入子图像的任意外画尺寸。我们的实验表明，我们提出的方法可以产生视觉上有吸引力的结果，广义的图像外画对国家的最先进的图像外画方法。
英文摘要：
In this paper, we develop a novel transformer-based generative adversarial neural network called U-Transformer for generalised image outpainting problem. Different from most present image outpainting methods conducting horizontal extrapolation, our generalised image outpainting could extrapolate visual context all-side around a given image with plausible structure and details even for complicated scenery, building, and art images. Specifically, we design a generator as an encoder-to-decoder structure embedded with the popular Swin Transformer blocks. As such, our novel neural network can better cope with image long-range dependencies which are crucially important for generalised image outpainting. We propose additionally a U-shaped structure and multi-view Temporal Spatial Predictor (TSP) module to reinforce image self-reconstruction as well as unknown-part prediction smoothly and realistically. By adjusting the predicting step in the TSP module in the testing stage, we can generate arbitrary outpainting size given the input sub-image. We experimentally demonstrate that our proposed method could produce visually appealing results for generalized image outpainting against the state-of-the-art image outpainting approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
