RAFaRe：从伪2D和3D对学习鲁棒和精确的非参数3D人脸重建
RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs
摘要：
提出了一种鲁棒、精确的单视图三维人脸重建非参数方法。虽然人们已经在参数化SVFR上投入了大量的精力，但在最终的3D形状和地面实况之间仍然存在明显的差距。我们认为存在两个主要障碍：1）参数模型的表示仅限于特定的人脸数据库; 2）拟合数据集中的2D图像和3D形状明显不对齐。为了解决这些问题，通过首先渲染详细的3D面部，然后将野生图像中的面部与渲染的面部交换，来创建大规模的伪2D\& 3D数据集。这些伪2D和3D对是从公开可用的数据集创建的，这些数据集消除了2D和3D数据之间的差距，同时覆盖了不同的外观，姿势，场景和照明。我们进一步提出了一个非参数方案，从创建的数据集学习一个良好的广义SVFR模型，所提出的分层有符号距离函数在预测中等规模和小规模的3D面部几何形状方面是有效的。我们的模型在FaceScape野生/实验室和MICC基准测试中优于以前的方法，并且可以很好地推广到各种外观，姿势，表情和野外环境。代码发布于www.example.com。
英文摘要：
We propose a robust and accurate non-parametric method for single-view 3D face reconstruction (SVFR). While tremendous efforts have been devoted to parametric SVFR, a visible gap still lies between the result 3D shape and the ground truth. We believe there are two major obstacles: 1) the representation of the parametric model is limited to a certain face database; 2) 2D images and 3D shapes in the fitted datasets are distinctly misaligned. To resolve these issues, a large-scale pseudo 2D\&3D dataset is created by first rendering the detailed 3D faces, then swapping the face in the wild images with the rendered face. These pseudo 2D&3D pairs are created from publicly available datasets which eliminate the gaps between 2D and 3D data while covering diverse appearances, poses, scenes, and illumination. We further propose a non-parametric scheme to learn a well-generalized SVFR model from the created dataset, and the proposed hierarchical signed distance function turns out to be effective in predicting middle-scale and small-scale 3D facial geometry. Our model outperforms previous methods on FaceScape-wild/lab and MICC benchmarks and is well generalized to various appearances, poses, expressions, and in-the-wild environments. The code is released at http://github.com/zhuhao-nju/rafare .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
