基于单目RGB视频的精细化几何引导头部虚拟化身重建
Refined Geometry-guided Head Avatar Reconstruction from Monocular RGB Video
AI总结：
本文提出一种两阶段头部虚拟化身重建网络，利用精细3D网格表示和SDF细化，增强NeRF渲染，实现高保真重建，性能优于现有方法。
摘要：
从单目视频中高保真重建头部虚拟化身对于虚拟人应用非常理想，但在计算机图形学和计算机视觉领域仍是一个挑战。在本文中，我们提出了一种两阶段的头部虚拟化身重建网络，该网络结合了精细的3D网格表示。与现有方法依赖于从3DMM导出的粗糙模板3D表示不同，我们的方法旨在学习一种适合NeRF的精细网格表示，以捕捉复杂的面部细微差别。在第一阶段，我们使用初始网格训练3DMM存储的NeRF，以利用几何先验，并通过一组一致的潜在编码跨帧整合观测。在第二阶段，我们利用一种基于从初始NeRF的密度场构建的SDF的新型网格细化程序。为了减轻NeRF密度场中典型的噪声而不损害3DMM的特征，我们对位移场应用拉普拉斯平滑。随后，我们使用这些精细网格进行第二阶段训练，引导网络的学习过程捕捉精细的面部细节。我们的实验表明，我们的方法进一步增强了基于初始网格的NeRF渲染，并在使用此类输入重建高保真头部虚拟化身方面实现了优于最先进方法的性能。
英文摘要：
High-fidelity reconstruction of head avatars from monocular videos is highly desirable for virtual human applications, but it remains a challenge in the fields of computer graphics and computer vision. In this paper, we propose a two-phase head avatar reconstruction network that incorporates a refined 3D mesh representation. Our approach, in contrast to existing methods that rely on coarse template-based 3D representations derived from 3DMM, aims to learn a refined mesh representation suitable for a NeRF that captures complex facial nuances. In the first phase, we train 3DMM-stored NeRF with an initial mesh to utilize geometric priors and integrate observations across frames using a consistent set of latent codes. In the second phase, we leverage a novel mesh refinement procedure based on an SDF constructed from the density field of the initial NeRF. To mitigate the typical noise in the NeRF density field without compromising the features of the 3DMM, we employ Laplace smoothing on the displacement field. Subsequently, we apply a second-phase training with these refined meshes, directing the learning process of the network towards capturing intricate facial details. Our experiments demonstrate that our method further enhances the NeRF rendering based on the initial mesh and achieves performance superior to state-of-the-art methods in reconstructing high-fidelity head avatars with such input.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
