arXiv 2109.145252023-06-26cs.CV

DRAN：用于条件图像合成的详细区域自适应归一化

DRAN: Detailed Region-Adaptive Normalization for Conditional Image Synthesis

Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；ByteDance Inc（字节跳动公司）

Yueming Lyu, Peibin Chen, Jingna Sun, Bo Peng, Xu Wang, Jing Dong

备注：: Accepted by TMM 2023

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对条件图像合成难以处理细粒度风格的问题，提出新型归一化模块DRAN，通过多级结构和动态门控自适应融合粗细粒度特征，在化妆迁移等任务上显著提升基线模型表现。

摘要：

近年来，条件图像合成因其在图像生成过程中的可控性而受到越来越多的关注。尽管近期的研究取得了逼真的效果，但大多数方法在处理具有微妙细节的细粒度风格时仍存在困难。为了解决这个问题，本文提出了一种名为 Detailed Region-Adaptive Normalization (DRAN) 的新型归一化模块。它能够自适应地学习细粒度和粗粒度风格表示。具体而言，我们首先引入了一个多级结构 Spatiality-aware Pyramid Pooling，以指导模型学习从粗到细的特征。然后，为了自适应地融合不同级别的风格，我们提出了 Dynamic Gating，使其能够根据不同的空间区域自适应地融合不同级别的风格。最后，我们收集了一个新的化妆数据集（Makeup-Complex dataset），其中包含各种复杂的化妆风格以及多样的姿势和表情。为了评估方法的有效性并展示其通用性，我们在化妆迁移和语义图像合成上进行了一系列实验。定量和定性实验表明，配备了 DRAN 的简单基线模型能够在复杂风格迁移和详细纹理合成中取得显著的改进。代码和所提出的数据集将在 https://github.com/Yueming6568/DRAN-makeup.git 上提供。

英文摘要：

In recent years, conditional image synthesis has attracted growing attention due to its controllability in the image generation process. Although recent works have achieved realistic results, most of them have difficulty handling fine-grained styles with subtle details. To address this problem, a novel normalization module, named Detailed Region-Adaptive Normalization~(DRAN), is proposed. It adaptively learns both fine-grained and coarse-grained style representations. Specifically, we first introduce a multi-level structure, Spatiality-aware Pyramid Pooling, to guide the model to learn coarse-to-fine features. Then, to adaptively fuse different levels of styles, we propose Dynamic Gating, making it possible to adaptively fuse different levels of styles according to different spatial regions. Finally, we collect a new makeup dataset (Makeup-Complex dataset) that contains a wide range of complex makeup styles with diverse poses and expressions. To evaluate the effectiveness and show the general use of our method, we conduct a set of experiments on makeup transfer and semantic image synthesis. Quantitative and qualitative experiments show that equipped with DRAN, simple baseline models are able to achieve promising improvements in complex style transfer and detailed texture synthesis. Both the code and the proposed dataset will be available at https://github.com/Yueming6568/DRAN-makeup.git.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。