arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.14078cs.CV
收藏

Stable-Hair:基于扩散模型的真实世界发型迁移

Stable-Hair: Real-World Hair Transfer via Diffusion Model

Shanghai Jiao Tong University(上海交通大学);Tiamat AI;Ocean University of China(中国海洋大学);National University of Singapore(新加坡国立大学);Peking University(北京大学);Shenyang Institute of Automation Chinese Academy of Sciences(中国科学院沈阳自动化研究所)

Yuxuan Zhang, Qing Zhang, Yiren Song, Jichao Zhang, Hao Tang, Jiaming Liu

AI总结:

针对真实世界复杂发型迁移难题,本文提出两阶段扩散框架 Stable-Hair,通过 Bald Converter、Hair Extractor、Latent IdentityNet 与 Latent ControlNet 实现高细节、高保真且保持身份背景的虚拟试发。

摘要:

当前的发型迁移方法难以处理多样且复杂的发型,限制了其在真实世界场景中的适用性。本文提出一种新颖的基于扩散的发型迁移框架,名为 Stable-Hair,它能够稳健地将广泛的真实世界发型迁移到用户提供的人脸,以实现虚拟发型试戴。为实现这一目标,Stable-Hair 框架被设计为两阶段流水线。第一阶段,我们与 stable diffusion 一起训练 Bald Converter,以从用户提供的人脸图像中移除头发,生成秃头图像。第二阶段,我们专门设计了 Hair Extractor 和 Latent IdentityNet,将目标发型以高度细节和高保真度迁移到秃头图像。Hair Extractor 经过训练,可编码具有所需发型的参考图像;Latent IdentityNet 则确保身份和背景的一致性。为最小化源图像与迁移结果之间的颜色偏差,我们引入新颖的 Latent ControlNet 架构,该架构同时充当 Bald Converter 和 Latent IdentityNet。在我们整理的三元组数据集上训练后,该方法能够将高度细节化且高保真的发型准确迁移到源图像。大量实验表明,与现有发型迁移方法相比,本方法取得了最先进的性能。项目页面:https://xiaojiu-z.github.io/Stable-Hair.github.io/

英文摘要:

Current hair transfer methods struggle to handle diverse and intricate hairstyles, limiting their applicability in real-world scenarios. In this paper, we propose a novel diffusion-based hair transfer framework, named \textit{Stable-Hair}, which robustly transfers a wide range of real-world hairstyles to user-provided faces for virtual hair try-on. To achieve this goal, our Stable-Hair framework is designed as a two-stage pipeline. In the first stage, we train a Bald Converter alongside stable diffusion to remove hair from the user-provided face images, resulting in bald images. In the second stage, we specifically designed a Hair Extractor and a Latent IdentityNet to transfer the target hairstyle with highly detailed and high-fidelity to the bald image. The Hair Extractor is trained to encode reference images with the desired hairstyles, while the Latent IdentityNet ensures consistency in identity and background. To minimize color deviations between source images and transfer results, we introduce a novel Latent ControlNet architecture, which functions as both the Bald Converter and Latent IdentityNet. After training on our curated triplet dataset, our method accurately transfers highly detailed and high-fidelity hairstyles to the source images. Extensive experiments demonstrate that our approach achieves state-of-the-art performance compared to existing hair transfer methods. Project page: \textcolor{red}{\url{https://xiaojiu-z.github.io/Stable-Hair.github.io/}}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑