arXiv 2203.059082022-03-11cs.CV

BabyNet：从未校准的照片重建婴儿的3D面部

BabyNet: Reconstructing 3D faces of babies from uncalibrated photographs

Pompeu Fabra University（庞培法布拉大学）；Colorado School of Public Health（科罗拉多公共卫生学院）；University of Colorado Anschutz Medical Campus（科罗拉多大学安舒茨医学校区）；Children’s Hospital Colorado（科罗拉多儿童医院）；Children’s National Hospital（国家儿童医院）；George Washington University School of Medicine and Health Sciences（乔治华盛顿大学医学与健康科学学院）

Araceli Morales, Antonio R. Porras, Marius George Linguraru, Gemma Piella, Federico M. Sukno

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

摘要：

我们提出了一个三维人脸重建系统，旨在从未校准的照片，BabyNet恢复婴儿的三维面部几何形状。由于婴儿的3D面部几何形状与成人的面部几何形状有很大不同，因此需要婴儿特定的面部重建系统。BabyNet由两个阶段组成：1）3D图形卷积自动编码器学习婴儿3D面部形状的潜在空间; 2）2D编码器，基于使用迁移学习提取的代表性特征将照片映射到3D潜在空间。通过这种方式，使用预训练的3D解码器，我们可以从2D图像中恢复3D人脸。我们对BabyNet进行了评估，结果表明：1）基于成人数据集的方法无法对婴儿的3D面部几何形状进行建模，这证明了婴儿特定方法的必要性; 2）即使使用婴儿特定的3D变形模型（如BabyFM），BabyNet也优于经典的模型拟合方法。

英文摘要：

We present a 3D face reconstruction system that aims at recovering the 3D facial geometry of babies from uncalibrated photographs, BabyNet. Since the 3D facial geometry of babies differs substantially from that of adults, baby-specific facial reconstruction systems are needed. BabyNet consists of two stages: 1) a 3D graph convolutional autoencoder learns a latent space of the baby 3D facial shape; and 2) a 2D encoder that maps photographs to the 3D latent space based on representative features extracted using transfer learning. In this way, using the pre-trained 3D decoder, we can recover a 3D face from 2D images. We evaluate BabyNet and show that 1) methods based on adult datasets cannot model the 3D facial geometry of babies, which proves the need for a baby-specific method, and 2) BabyNet outperforms classical model-fitting methods even when a baby-specific 3D morphable model, such as BabyFM, is used.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。