arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.09692cs.CV
收藏

基于隐式表示的人脸几何细节恢复

Facial Geometric Detail Recovery via Implicit Representation

Shanghai Jiao Tong University(上海交通大学);Imperial College London(帝国理工学院);Huawei CBG(华为消费者业务集团)

Xingyu Ren, Alexandros Lattas, Baris Gecer, Jiankang Deng, Chao Ma, Xiaokang Yang, Stefanos Zafeiriou

摘要:

从单个面部图像中学习具有精细尺度细节的密集3D模型是非常具有挑战性和不适定的。为了解决这个问题,许多方法通过面部先验拟合平滑几何形状,同时学习细节作为额外的位移图或个性化基础。然而,这些技术通常需要大量成对多视图数据或3D扫描的数据集,而这样的数据集是稀缺且昂贵的。为了减轻沉重的数据依赖,我们提出了一个强大的纹理引导的几何细节恢复方法,只使用一个单一的野生面部图像。更具体地说,我们的方法结合了高质量的纹理完成与隐式曲面的强大表现力。最初,我们修补被遮挡的面部部分,生成完整的纹理,并构建同一主题的精确多视图数据集。为了估计详细的几何形状,我们定义了一个隐式的有符号距离函数,并采用基于物理的隐式渲染器从生成的多视图图像中重建精细的几何细节。我们的方法不仅恢复准确的面部细节,而且还分解法线,阴影和阴影部分在一个自我监督的方式。最后,我们将隐式形状细节注册到3D Morphable Model模板,该模板可用于传统的建模和渲染管道。大量的实验表明,所提出的方法可以从单个图像中重建令人印象深刻的面部细节,特别是与在大型数据集上训练的最先进的方法相比。

英文摘要:

Learning a dense 3D model with fine-scale details from a single facial image is highly challenging and ill-posed. To address this problem, many approaches fit smooth geometries through facial prior while learning details as additional displacement maps or personalized basis. However, these techniques typically require vast datasets of paired multi-view data or 3D scans, whereas such datasets are scarce and expensive. To alleviate heavy data dependency, we present a robust texture-guided geometric detail recovery approach using only a single in-the-wild facial image. More specifically, our method combines high-quality texture completion with the powerful expressiveness of implicit surfaces. Initially, we inpaint occluded facial parts, generate complete textures, and build an accurate multi-view dataset of the same subject. In order to estimate the detailed geometry, we define an implicit signed distance function and employ a physically-based implicit renderer to reconstruct fine geometric details from the generated multi-view images. Our method not only recovers accurate facial details but also decomposes normals, albedos, and shading parts in a self-supervised way. Finally, we register the implicit shape details to a 3D Morphable Model template, which can be used in traditional modeling and rendering pipelines. Extensive experiments demonstrate that the proposed approach can reconstruct impressive facial details from a single image, especially when compared with state-of-the-art methods trained on large datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑