arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.01094cs.CV
收藏

HairFastGAN:基于快速编码器方法的逼真且稳健的头发迁移

HairFastGAN: Realistic and Robust Hair Transfer with a Fast Encoder-Based Approach

HSE University(高等经济大学);Skolkovo Institute of Science and Technology(斯科尔科沃科学技术研究所);AIRI(人工智能研究院);Constructor University(康斯特大学)

Maxim Nikolaev, Mikhail Kuznetsov, Dmitry Vetrov, Aibek Alanov

AI总结:

提出HairFast模型,通过FS潜空间架构和增强编码器,实现快速、高质量、姿态鲁棒的发型迁移,性能优于优化方法。

摘要:

我们的论文解决了将参考图像中的发型迁移到输入照片以进行虚拟试发的复杂任务。由于需要适应各种照片姿态、发型的敏感性以及缺乏客观指标,这项任务具有挑战性。当前最先进的发型迁移方法对方法的不同部分使用优化过程,使其速度慢得不可原谅。同时,基于编码器的更快模型质量非常低,因为它们要么在StyleGAN的W+空间中操作,要么使用其他低维图像生成器。此外,当源姿态与目标姿态差异很大时,两种方法都存在发型迁移问题,因为它们要么完全不考虑姿态,要么处理效率低下。在我们的论文中,我们提出了HairFast模型,该模型独特地解决了这些问题,并实现了高分辨率、接近实时的性能以及优于基于优化问题的方法的重建质量。我们的解决方案包括在StyleGAN的FS潜空间中运行的新架构、增强的修复方法、改进的用于更好对齐和颜色迁移的编码器,以及用于后处理的新编码器。我们方法的效果通过在随机发型迁移后的真实感指标以及迁移原始发型时的重建指标得到证明。在从不同图像迁移发型的形状和颜色的最困难场景中,我们的方法在Nvidia V100上不到一秒内完成。我们的代码可在https://github.com/AIRI-Institute/HairFastGAN获取。

英文摘要:

Our paper addresses the complex task of transferring a hairstyle from a reference image to an input photo for virtual hair try-on. This task is challenging due to the need to adapt to various photo poses, the sensitivity of hairstyles, and the lack of objective metrics. The current state of the art hairstyle transfer methods use an optimization process for different parts of the approach, making them inexcusably slow. At the same time, faster encoder-based models are of very low quality because they either operate in StyleGAN's W+ space or use other low-dimensional image generators. Additionally, both approaches have a problem with hairstyle transfer when the source pose is very different from the target pose, because they either don't consider the pose at all or deal with it inefficiently. In our paper, we present the HairFast model, which uniquely solves these problems and achieves high resolution, near real-time performance, and superior reconstruction compared to optimization problem-based methods. Our solution includes a new architecture operating in the FS latent space of StyleGAN, an enhanced inpainting approach, and improved encoders for better alignment, color transfer, and a new encoder for post-processing. The effectiveness of our approach is demonstrated on realism metrics after random hairstyle transfer and reconstruction when the original hairstyle is transferred. In the most difficult scenario of transferring both shape and color of a hairstyle from different images, our method performs in less than a second on the Nvidia V100. Our code is available at https://github.com/AIRI-Institute/HairFastGAN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑