用于可控人物图像生成的解耦表征学习
Disentangled Representation Learning for Controllable Person Image Generation
AI总结：
提出 DRL-CPG 框架，通过 transformer 属性编码器、课程学习、随机掩码无关策略和 DAD 残差解码器学习解耦表征，实现可控的人物姿态与属性迁移。
摘要：
本文提出了一个名为 DRL-CPG 的新颖框架，用于学习解耦的潜在表征，以实现可控人物图像生成；该框架能够生成逼真的人物图像，并使其具有由不同来源人物提供的期望姿态和人体属性（例如姿态、头部、上衣和裤子）。不同于现有工作利用语义掩码获得各组件表征的做法，本文提出通过一种新颖的、基于 transformer 的属性编码器来生成解耦的潜在代码，该编码器以课程学习方式训练，即从相对简单的步骤逐步过渡到逐渐困难的步骤。文中引入了一种随机组件掩码无关策略，从人物分割掩码中随机移除组件掩码，其目的是提高训练难度，并促使 transformer 编码器识别各组件之间的潜在边界。这使模型能够同时迁移组件的形状和纹理。此外，本文提出了一种新颖的属性解码器网络，通过设计良好的双重自适应反归一化（Dual Adaptive Denormalization，DAD）残差块来整合多层次属性（例如结构特征和属性表征）。大量实验有力地表明，所提方法能够同时迁移不同人体部位的纹理和形状，并产生逼真的结果。据我们所知，我们是首个将 transformer 用于学习人物图像生成的解耦潜在表征的工作。
英文摘要：
In this paper, we propose a novel framework named DRL-CPG to learn disentangled latent representation for controllable person image generation, which can produce realistic person images with desired poses and human attributes (e.g., pose, head, upper clothes, and pants) provided by various source persons. Unlike the existing works leveraging the semantic masks to obtain the representation of each component, we propose to generate disentangled latent code via a novel attribute encoder with transformers trained in a manner of curriculum learning from a relatively easy step to a gradually hard one. A random component mask-agnostic strategy is introduced to randomly remove component masks from the person segmentation masks, which aims at increasing the difficulty of training and promoting the transformer encoder to recognize the underlying boundaries between each component. This enables the model to transfer both the shape and texture of the components. Furthermore, we propose a novel attribute decoder network to integrate multi-level attributes (e.g., the structure feature and the attribute representation) with well-designed Dual Adaptive Denormalization (DAD) residual blocks. Extensive experiments strongly demonstrate that the proposed approach is able to transfer both the texture and shape of different human parts and yield realistic results. To our knowledge, we are the first to learn disentangled latent representations with transformers for person image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
