可控的多域语义艺术作品合成
Controllable Multi-domain Semantic Artwork Synthesis
AI总结：
本文针对语义艺术合成缺少分割数据集的问题，构建 ArtSem 数据集，并提出基于域相关变分编码器与 SSTAN 的多域 GAN 合成模型，实现更高质量且可控的艺术图像生成。
摘要：
我们提出了一个从语义布局进行艺术作品多域合成的新颖框架。这一具有挑战性的任务的主要局限之一是缺乏用于艺术合成的公开可用分割数据集。为了解决这一问题，我们提出了一个名为 ArtSem 的数据集，其中包含来自4个不同域的40,000张艺术作品图像及其对应的语义标签图。我们构建该数据集的方式是：首先从风景摄影中提取语义图，随后提出一种基于条件生成对抗网络（GAN）的方法，无需成对训练数据即可从语义图生成高质量艺术作品。此外，我们提出了一种艺术作品合成模型，该模型使用依赖于域的变分编码器来实现高质量多域合成。该模型通过一种简单而有效的归一化方法得到改进和补充，该方法基于对语义和风格进行联合归一化，我们称之为空间风格自适应归一化（Spatially STyle-Adaptive Normalization，SSTAN）。与以往仅将语义布局作为输入的方法不同，我们的模型能够学习风格与语义信息的联合表示，从而在合成艺术图像时获得更好的生成质量。结果表明，我们的模型学会了在潜在空间中分离各个域，因此，通过识别分隔不同域的超平面，我们还可以对合成的艺术作品执行细粒度控制。通过结合我们提出的数据集和方法，我们能够生成比现有方法质量更高、可由用户控制的艺术作品。
英文摘要：
We present a novel framework for multi-domain synthesis of artwork from semantic layouts. One of the main limitations of this challenging task is the lack of publicly available segmentation datasets for art synthesis. To address this problem, we propose a dataset, which we call ArtSem, that contains 40,000 images of artwork from 4 different domains with their corresponding semantic label maps. We generate the dataset by first extracting semantic maps from landscape photography and then propose a conditional Generative Adversarial Network (GAN)-based approach to generate high-quality artwork from the semantic maps without necessitating paired training data. Furthermore, we propose an artwork synthesis model that uses domain-dependent variational encoders for high-quality multi-domain synthesis. The model is improved and complemented with a simple but effective normalization method, based on normalizing both the semantic and style jointly, which we call Spatially STyle-Adaptive Normalization (SSTAN). In contrast to previous methods that only take semantic layout as input, our model is able to learn a joint representation of both style and semantic information, which leads to better generation quality for synthesizing artistic images. Results indicate that our model learns to separate the domains in the latent space, and thus, by identifying the hyperplanes that separate the different domains, we can also perform fine-grained control of the synthesized artwork. By combining our proposed dataset and approach, we are able to generate user-controllable artwork that is of higher quality than existing
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
