通过扩散模型进行语义图像合成
Semantic Image Synthesis via Diffusion Models
AI总结：
本文提出基于DDPM的语义图像合成框架，通过改进的采样策略和网络结构提升生成质量和语义可解释性。
摘要：
去噪扩散概率模型（DDPMs）在各种图像生成任务中相比生成对抗网络（GANs）取得了显著的成功。最近关于语义图像合成的工作主要遵循事实上的基于GAN的方法，这可能导致生成图像的质量或多样性不令人满意。在本文中，我们提出了一种基于DDPM的新框架用于语义图像合成。与之前的条件扩散模型直接将语义布局和噪声图像作为输入馈送到U-Net结构不同，我们的框架对语义布局和噪声图像进行了不同的处理。它将噪声图像馈送到U-Net结构的编码器，而将语义布局馈送到解码器，通过多层空间自适应归一化操作符。为了进一步提高语义图像合成中的生成质量和语义可解释性，我们引入了分类器自由引导采样策略，该策略在采样过程中承认无条件模型的评分。在四个基准数据集上的广泛实验证明了我们提出的方法的有效性，在忠实度（FID）和多样性（LPIPS）方面实现了最先进的性能。我们的代码和预训练模型可在https://github.com/WeilunWang/semantic-diffusion-model上获得。
英文摘要：
Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks compared with Generative Adversarial Nets (GANs). Recent work on semantic image synthesis mainly follows the de facto GAN-based approaches, which may lead to unsatisfactory quality or diversity of generated images. In this paper, we propose a novel framework based on DDPM for semantic image synthesis. Unlike previous conditional diffusion model directly feeds the semantic layout and noisy image as input to a U-Net structure, which may not fully leverage the information in the input semantic mask, our framework processes semantic layout and noisy image differently. It feeds noisy image to the encoder of the U-Net structure while the semantic layout to the decoder by multi-layer spatially-adaptive normalization operators. To further improve the generation quality and semantic interpretability in semantic image synthesis, we introduce the classifier-free guidance sampling strategy, which acknowledge the scores of an unconditional model for sampling process. Extensive experiments on four benchmark datasets demonstrate the effectiveness of our proposed method, achieving state-of-the-art performance in terms of fidelity (FID) and diversity (LPIPS). Our code and pretrained models are available at https://github.com/WeilunWang/semantic-diffusion-model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
