arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.02550cs.CV
收藏

用于少样本生成式域适应的域重新调制

Domain Re-Modulation for Few-Shot Generative Domain Adaptation

University of Science and Technology of China(中国科学技术大学);JD Explore Academy(京东探索研究院)

Yi Wu, Ziqiang Li, Chaoyue Wang, Heliang Zheng, Shanshan Zhao, Bin Li, Dacheng Tao

AI总结:

本研究提出了一种名为域重新调制(DoRM)的创新生成器结构,通过冻结源生成器并引入新的映射和仿射模块来捕获目标域属性,同时结合基于相似性的结构损失保持跨域一致性,在少样本生成式域适应任务中实现了高保真的多域和混合域生成。

摘要:

在本研究中,我们深入探讨了少样本生成式域适应(GDA)任务,该任务涉及仅使用少量参考图像将预训练生成器从一个域转移到新域。受人类大脑在新领域获取知识方式的启发,我们提出了一种名为域重新调制(DoRM)的创新生成器结构。DoRM不仅满足了先前GDA研究所实现的高质量、大合成多样性和跨域一致性的标准,还融入了记忆和域关联,类似于人类大脑的运作方式。具体而言,DoRM冻结源生成器并引入新的映射和仿射模块(M&A模块),以在GDA期间捕获目标域的属性。此过程类似于人类大脑中新突触的形成。因此,在风格空间中发生线性可组合的域偏移。通过引入多个新的M&A模块,生成器获得了执行高保真多域和混合域生成的能力。此外,为了更有效地保持跨域一致性,我们引入了基于相似性的结构损失。该损失在训练期间将目标图像的自相关图与源图像对应的自相关图对齐。通过大量实验,我们在定量和定性两方面证明了我们的DoRM和基于相似性的结构损失在少样本GDA中的优越性能。代码将在https://github.com/wuyi2020/DoRM提供。

英文摘要:

In this study, we delve into the task of few-shot Generative Domain Adaptation (GDA), which involves transferring a pre-trained generator from one domain to a new domain using only a few reference images. Inspired by the way human brains acquire knowledge in new domains, we present an innovative generator structure called Domain Re-Modulation (DoRM). DoRM not only meets the criteria of high quality, large synthesis diversity, and cross-domain consistency, which were achieved by previous research in GDA, but also incorporates memory and domain association, akin to how human brains operate. Specifically, DoRM freezes the source generator and introduces new mapping and affine modules (M&A modules) to capture the attributes of the target domain during GDA. This process resembles the formation of new synapses in human brains. Consequently, a linearly combinable domain shift occurs in the style space. By incorporating multiple new M&A modules, the generator gains the capability to perform high-fidelity multi-domain and hybrid-domain generation. Moreover, to maintain cross-domain consistency more effectively, we introduce a similarity-based structure loss. This loss aligns the auto-correlation map of the target image with its corresponding auto-correlation map of the source image during training. Through extensive experiments, we demonstrate the superior performance of our DoRM and similarity-based structure loss in few-shot GDA, both quantitatively and qualitatively. The code will be available at https://github.com/wuyi2020/DoRM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑