RealDenseFace：基于密集UV空间先验的实时单目3D人脸重建

中文摘要 AI 辅助

近期的单目3D人脸重建方法通过将3D Morphable Model（3DMM）拟合至网络预测的密集先验，实现了高保真度，但优化阶段计算成本高昂，每张图像通常需要数十秒。本文提出RealDenseFace，一种基于优化的实时3D人脸重建方法，采用密集UV空间网络预测。核心思路是将3DMM拟合问题表述为非线性最小二乘问题，并通过定制的高斯-牛顿求解器求解，仅需数次迭代即可收敛。重建分为两个阶段：第一阶段，网络从单张RGB图像预测两张密集UV空间图，分别是用于UV到图像对齐的对应图，以及用于视向几何约束的相对深度图；第二阶段，求解器拟合从这些图中按顶点UV坐标采样得到的逐顶点目标。该求解器支持三种重建场景：单图像拟合、离线序列重建和在线跟踪。本文方法在NeRSemble SVFR基准上达到了顶尖精度，在线跟踪器帧率达80 FPS以上，离线序列重建速度较以往基于优化的基线快20倍以上。

英文摘要

Recent monocular 3D face reconstruction methods achieve high fidelity by fitting a 3D Morphable Model (3DMM) to dense priors predicted by networks, but the optimization stage is computationally expensive, often taking tens of seconds per image. We present RealDenseFace, a real-time optimization-based 3D face reconstruction method with dense UV-space network predictions. Our key idea is to formulate 3DMM fitting as a nonlinear least-squares problem and solve it with a tailored Gauss-Newton solver that converges in only a few iterations. The reconstruction is conducted in two stages. In the first stage, the network predicts two dense UV-space maps from a single RGB image: a correspondence map for UV-to-image alignment, and a relative-depth map for geometric constraints along the viewing direction. In the second stage, the solver fits per-vertex targets sampled from these maps at the vertex UV coordinates. The solver supports all three reconstruction settings: single-image fitting, offline sequence reconstruction, and online tracking. Our method achieves state-of-the-art accuracy on the NeRSemble SVFR benchmark. The online tracker runs at 80+ FPS, and the offline sequence reconstruction is over 20 times faster than previous optimization-based baselines.

发表机构

• Zhejiang University（浙江大学）

机构由 AI 辅助整理，请以论文原文为准。