arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.05142cs.CVcs.GR
收藏

HairCLIP:通过文字和参考图片设计你的头发

HairCLIP: Design Your Hair by Text and Reference Image

University of Science and Technology of China(中国科学技术大学);Microsoft Cloud AI(微软云人工智能);City University of Hong Kong(香港城市大学)

Tianyi Wei, Dongdong Chen, Wenbo Zhou, Jing Liao, Zhentao Tan, Lu Yuan, Weiming Zhang, Nenghai Yu

摘要:

头发编辑是计算机视觉和图形学中一个有趣且具有挑战性的问题。许多现有的方法需要绘制良好的草图或遮罩作为编辑的条件输入,然而这些交互既不简单也不高效。为了将用户从繁琐的交互过程中解放出来,提出了一种新的头发编辑交互模式,可以根据用户提供的文本或参考图片,单独或联合操作头发属性。为此,我们在一个共享的嵌入空间中编码图像和文本条件,并提出了一个统一的头发编辑框架,利用强大的图像文本表示能力的对比图像预训练(CLIP)模型。通过精心设计的网络结构和损失函数,我们的框架可以以一种解脱的方式执行高质量的头发编辑。大量的实验表明,我们的方法在操作精度,编辑结果的视觉现实主义,和不相关的属性保存方面的优越性。项目回购是https://github.com/wty-ustc/HairCLIP。

英文摘要:

Hair editing is an interesting and challenging problem in computer vision and graphics. Many existing methods require well-drawn sketches or masks as conditional inputs for editing, however these interactions are neither straightforward nor efficient. In order to free users from the tedious interaction process, this paper proposes a new hair editing interaction mode, which enables manipulating hair attributes individually or jointly based on the texts or reference images provided by users. For this purpose, we encode the image and text conditions in a shared embedding space and propose a unified hair editing framework by leveraging the powerful image text representation capability of the Contrastive Language-Image Pre-Training (CLIP) model. With the carefully designed network structures and loss functions, our framework can perform high-quality hair editing in a disentangled manner. Extensive experiments demonstrate the superiority of our approach in terms of manipulation accuracy, visual realism of editing results, and irrelevant attribute preservation. Project repo is https://github.com/wty-ustc/HairCLIP.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑