arXiv 2307.147702025-07-09cs.CV

3DPortraitGAN：从包含多样身体姿态的单视图肖像数据集中学习四分之一头像3D GAN

3DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Poses

State Key Lab of CAD\&CG, Zhejiang University（浙江大学CAD与计算机图形学国家重点实验室）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Hong Kong University of Science（香港科学大学）；ZJU-Tencent Game（浙大腾讯游戏与智能图形创新技术联合实验室）；Intelligent Graphics Innovation Technology Joint Lab（智能图形创新技术联合实验室）

Yiqian Wu, Hao Xu, Xiangjun Tang, Yue Shangguan, Hongbo Fu, Xiaogang Jin

备注：: Accepted for publication in IEEE Transactions on Circuits and Systems for Video Technology

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对现有3D人脸生成器无法生成完整四分之一头像的问题，提出3DPortraitGAN，利用新数据集360°PHQ和身体姿态自学习，从单视图数据生成全角度视图一致且几何完整的3D肖像。

摘要：

3D感知的人脸生成器通常在主要由近正面人脸数据组成的2D现实生活人脸图像数据集上训练，因此，它们无法构建具有完整头部、颈部和肩部几何结构的四分之一头像3D肖像。造成这一问题的原因有两个：首先，现有的人脸识别方法难以从大相机角度或背面视图提取人脸数据。其次，由于多样身体姿态引起的显著几何变形，从单视图数据学习覆盖四分之一头像区域的3D肖像分布具有挑战性。为此，我们首先创建了数据集360°-Portrait-HQ（简称360°PHQ），该数据集包含高质量的单视图真实肖像，并标注了各种相机参数（偏航角覆盖整个360°范围）和身体姿态。然后，我们提出了3DPortraitGAN，这是第一个3D感知的四分之一头像肖像生成器，它通过身体姿态自学习从360°PHQ数据集中学习规范化的3D头像分布。我们的模型可以从所有相机角度生成视图一致的肖像图像，并具有规范化的四分之一头像3D表示。我们的实验表明，所提出的框架能够准确预测肖像身体姿态，并从所有相机角度生成具有完整几何结构的视图一致、逼真的肖像图像。

英文摘要：

3D-aware face generators are typically trained on 2D real-life face image datasets that primarily consist of near-frontal face data, and as such, they are unable to construct one-quarter headshot 3D portraits with complete head, neck, and shoulder geometry. Two reasons account for this issue: First, existing facial recognition methods struggle with extracting facial data captured from large camera angles or back views. Second, it is challenging to learn a distribution of 3D portraits covering the one-quarter headshot region from single-view data due to significant geometric deformation caused by diverse body poses. To this end, we first create the dataset 360°-Portrait-HQ (360°PHQ for short) which consists of high-quality single-view real portraits annotated with a variety of camera parameters (the yaw angles span the entire 360° range) and body poses. We then propose 3DPortraitGAN, the first 3D-aware one-quarter headshot portrait generator that learns a canonical 3D avatar distribution from the 360°PHQ dataset with body pose self-learning. Our model can generate view-consistent portrait images from all camera angles with a canonical one-quarter headshot 3D representation. Our experiments show that the proposed framework can accurately predict portrait body poses and generate view-consistent, realistic portrait images with complete geometry from all camera angles.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。