arXiv 2503.117922025-05-14cs.CV

StyleMorpheus：一种基于风格的3D感知可形变人脸模型

StyleMorpheus: A Style-Based 3D-Aware Morphable Face Model

Department of Electrical and Computer Engineering, The University of British Columbia（电气与计算机工程系，不列颠哥伦比亚大学）；Huawei Technologies Canada（华为技术加拿大公司）；Department of Computer Science, Mathematics, Physics and Statistics, The University of British Columbia (Okanagan)（计算机科学、数学、物理与统计学系，不列颠哥伦比亚大学（OKanagan））

Peizhi Yan, Rabab K. Ward, Dan Wang, Qiang Tang, Shan Du

备注：: 13 pages, work was completed in 2023

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出StyleMorpheus，一种可在野外图像上训练的基于风格的神经3DMM，通过自动编码器、风格代码与对抗微调实现解耦可控、照片级真实且实时的3D人脸重建与编辑。

摘要：

在3D人脸建模方面，近期发展的3D感知神经渲染方法能够渲染具有任意视角的照片级真实感人脸图像。然而，参数化可控3D感知人脸模型的训练仍依赖实验室采集的大规模数据集。为解决该问题，本文提出“StyleMorpheus”，这是首个基于风格的神经3D可形变人脸模型（3DMM），可在野外图像上训练。它继承了3DMM对人脸身份、表情和外观的解耦可控性，而无需精确重建的显式3D形状。StyleMorpheus采用自动编码器结构。编码器旨在学习具有代表性的解耦参数代码空间，解码器则在网络的不同子模块中利用与形状和外观相关的风格代码来改进解耦。此外，我们通过基于风格的生成对抗学习对解码器进行微调，以实现照片级逼真的3D渲染质量。所提出的基于风格的设计使StyleMorpheus取得了最先进的3D感知人脸重建结果，同时还允许对重建的人脸进行解耦控制。我们的模型实现了实时渲染速度，使其可用于虚拟现实应用。我们还展示了所提出的基于风格的设计在风格混合和颜色编辑等人脸编辑应用中的能力。项目主页：https://github.com/ubc-3d-vision-lab/StyleMorpheus。

英文摘要：

For 3D face modeling, the recently developed 3D-aware neural rendering methods are able to render photorealistic face images with arbitrary viewing directions. The training of the parametric controllable 3D-aware face models, however, still relies on a large-scale dataset that is lab-collected. To address this issue, this paper introduces "StyleMorpheus", the first style-based neural 3D Morphable Face Model (3DMM) that is trained on in-the-wild images. It inherits 3DMM's disentangled controllability (over face identity, expression, and appearance) but without the need for accurately reconstructed explicit 3D shapes. StyleMorpheus employs an auto-encoder structure. The encoder aims at learning a representative disentangled parametric code space and the decoder improves the disentanglement using shape and appearance-related style codes in the different sub-modules of the network. Furthermore, we fine-tune the decoder through style-based generative adversarial learning to achieve photorealistic 3D rendering quality. The proposed style-based design enables StyleMorpheus to achieve state-of-the-art 3D-aware face reconstruction results, while also allowing disentangled control of the reconstructed face. Our model achieves real-time rendering speed, allowing its use in virtual reality applications. We also demonstrate the capability of the proposed style-based design in face editing applications such as style mixing and color editing. Project homepage: https://github.com/ubc-3d-vision-lab/StyleMorpheus.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。