基于无条件生成器的语义图像合成
Semantic Image Synthesis with Unconditional Generator
AI总结：
本文提出利用预训练无条件生成器，通过聚类特征图得到代理掩码并重排特征，再结合语义映射器实现无需大规模语义分割数据集的语义图像合成，可应用于空间编辑、草图到照片等任务。
摘要：
语义图像合成（SIS）旨在生成与给定语义掩码相匹配的真实图像。尽管近期进展已能实现高质量结果和精确的空间控制，但这些方法需要大规模的语义分割数据集来训练模型。相反，我们提出采用一个预训练的无条件生成器，并根据代理掩码重新排列其特征图。代理掩码通过对生成器中随机样本的特征图进行简单聚类来准备。特征重排器学习重新排列原始特征图，以匹配来自原始样本本身或随机样本的代理掩码的形状。随后，我们引入一个语义映射器，从包括语义掩码在内的各种输入条件中生成代理掩码。我们的方法在多种应用中具有通用性，例如真实图像的自由形式空间编辑、草图到照片，甚至涂鸦到照片。实验在多个数据集上验证了我们方法的优势：人脸、动物脸和建筑物。
英文摘要：
Semantic image synthesis (SIS) aims to generate realistic images that match given semantic masks. Despite recent advances allowing high-quality results and precise spatial control, they require a massive semantic segmentation dataset for training the models. Instead, we propose to employ a pre-trained unconditional generator and rearrange its feature maps according to proxy masks. The proxy masks are prepared from the feature maps of random samples in the generator by simple clustering. The feature rearranger learns to rearrange original feature maps to match the shape of the proxy masks that are either from the original sample itself or from random samples. Then we introduce a semantic mapper that produces the proxy masks from various input conditions including semantic masks. Our method is versatile across various applications such as free-form spatial editing of real images, sketch-to-photo, and even scribble-to-photo. Experiments validate advantages of our method on a range of datasets: human faces, animal faces, and buildings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
