arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.17272cs.CVcs.GRcs.LG
收藏

MyStyle:个性化生成先验

MyStyle: A Personalized Generative Prior

Google Research(谷歌研究院);Tel-Aviv University(特拉维夫大学)

Yotam Nitzan, Kfir Aberman, Qiurui He, Orly Liba, Michal Yarom, Yossi Gandelsman, Inbar Mosseri, Yael Pritch, Daniel Cohen-or

摘要:

我们介绍MyStyle,一个个性化的深度生成先验训练,用一个人的几个镜头。MyStyle允许重建,增强和编辑特定人的图像,以便输出忠实于人的关键面部特征。给定一个人的肖像图像的小参考集(~100),我们调整预训练的StyleGAN人脸生成器的权重,以在潜在空间中形成一个局部的,低维的,个性化的流形。我们发现,这个流形构成了一个个性化的区域,跨越潜在的代码与不同的肖像图像的个人。此外,我们证明了我们获得了个性化的生成先验,并提出了一种统一的方法,将其应用于各种不适定的图像增强问题,如修复和超分辨率,以及语义编辑。使用个性化的生成先验,我们获得了对输入图像表现出高保真度的输出,并且也忠实于参考集中个体的关键面部特征。我们证明了我们的方法与众多广泛认可的个人,我们有先验知识的预期结果的定性评价的合理使用的图像。我们评估我们的方法对少数镜头基线,并表明我们的个性化的先验,定量和定性,优于国家的最先进的替代品。

英文摘要:

We introduce MyStyle, a personalized deep generative prior trained with a few shots of an individual. MyStyle allows to reconstruct, enhance and edit images of a specific person, such that the output is faithful to the person's key facial characteristics. Given a small reference set of portrait images of a person (~100), we tune the weights of a pretrained StyleGAN face generator to form a local, low-dimensional, personalized manifold in the latent space. We show that this manifold constitutes a personalized region that spans latent codes associated with diverse portrait images of the individual. Moreover, we demonstrate that we obtain a personalized generative prior, and propose a unified approach to apply it to various ill-posed image enhancement problems, such as inpainting and super-resolution, as well as semantic editing. Using the personalized generative prior we obtain outputs that exhibit high-fidelity to the input images and are also faithful to the key facial characteristics of the individual in the reference set. We demonstrate our method with fair-use images of numerous widely recognizable individuals for whom we have the prior knowledge for a qualitative evaluation of the expected outcome. We evaluate our approach against few-shots baselines and show that our personalized prior, quantitatively and qualitatively, outperforms state-of-the-art alternatives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑