StyO：仅用单样本实现人脸风格化
StyO: Stylize Your Face in Only One-shot
AI总结：
针对单样本人脸风格化中几何变化与内容保留难兼顾的问题，提出基于LDM的StyO模型，通过IDL解耦内容风格、FCC精细控制重组，在多风格绘画上实现优于SOTA的效果。
摘要：
本文聚焦于单张艺术目标图像下的人脸风格化任务。现有该任务的方法在实现几何形态变化时，往往无法保留源图像内容。为此，我们提出了一种新型的StyO模型（即仅用单样本实现人脸风格化）来解决上述问题。具体而言，StyO采用解耦与重组策略：它首先将源图像和目标图像的内容与风格解耦为标识符，随后以交叉方式重组这些标识符，从而生成风格化人脸图像。通过这种方式，StyO将复杂图像分解为独立的特定属性，并将单样本人脸风格化简化为输入图像不同属性的组合，因此生成的结果能更好地匹配目标图像的人脸几何形态与源图像的内容。StyO基于潜在扩散模型（LDM）实现，由两个核心模块组成：1）标识符解耦学习器（IDL），用于解耦阶段。它将标识符表示为对比文本提示，即正向描述和负向描述；还引入了一种新型三重重建损失，对预训练的LDM进行微调，以将风格和内容编码到对应的标识符中。2）细粒度内容控制器（FCC），用于重组阶段。它重组从IDL得到的解耦标识符，形成增强文本提示以生成风格化人脸。此外，FCC还约束潜在特征与文本特征的交叉注意力图，以在结果中保留源人脸的细节。大量评估表明，StyO在多种风格的绘画作品上均能生成高质量图像，性能优于当前最优方法。
英文摘要：
This paper focuses on face stylization with a single artistic target. Existing works for this task often fail to retain the source content while achieving geometry variation. Here, we present a novel StyO model, ie. Stylize the face in only One-shot, to solve the above problem. In particular, StyO exploits a disentanglement and recombination strategy. It first disentangles the content and style of source and target images into identifiers, which are then recombined in a cross manner to derive the stylized face image. In this way, StyO decomposes complex images into independent and specific attributes, and simplifies one-shot face stylization as the combination of different attributes from input images, thus producing results better matching face geometry of target image and content of source one. StyO is implemented with latent diffusion models (LDM) and composed of two key modules: 1) Identifier Disentanglement Learner (IDL) for disentanglement phase. It represents identifiers as contrastive text prompts, ie. positive and negative descriptions. And it introduces a novel triple reconstruction loss to fine-tune the pre-trained LDM for encoding style and content into corresponding identifiers; 2) Fine-grained Content Controller (FCC) for the recombination phase. It recombines disentangled identifiers from IDL to form an augmented text prompt for generating stylized faces. In addition, FCC also constrains the cross-attention maps of latent and text features to preserve source face details in results. The extensive evaluation shows that StyO produces high-quality images on numerous paintings of various styles and outperforms the current state-of-the-art.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
