arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.14237cs.CVcs.AIcs.GR
收藏

基于仿射卷积网络的高质量3D人脸重建

High-Quality 3D Face Reconstruction with Affine Convolutional Networks

Zhejiang University(浙江大学);NetEase Fuxi AI Lab(网易伏羲人工智能实验室);Beihang University(北京航空航天大学)

Zhiqian Lin, Jiangke Lin, Lincheng Li, Yi Yuan, Zhengxia Zou

AI总结:

针对3D人脸重建中输入图像与UV输出空间错位的问题,提出Affine Convolution Networks学习仿射变换矩阵,无需参数和人工标注即可生成512x512高分辨率高质量UV图。

摘要:

近期基于卷积编码器-解码器架构和3DMM参数化的工作,在从单张输入图像进行正则视图重建方面展现出了巨大潜力。传统的CNN架构得益于利用输入和输出像素之间的空间对应关系。然而,在3D人脸重建中,输入图像(如人脸)与正则/UV输出之间的空间错位使得特征编码-解码过程极具挑战性。在本文中,为解决这一问题,我们提出了一种新的网络架构,即Affine Convolution Networks(仿射卷积网络),它使基于CNN的方法能够处理空间上不对应的输入和输出图像,同时保持高保真度的输出质量。在我们的方法中,通过仿射卷积层为特征图的每个空间位置学习一个仿射变换矩阵。此外,我们在UV空间中使用多个组件表示3D人头,包括用于纹理表示的漫反射图、用于几何表示的位置图,以及用于恢复现实世界中更复杂光照条件的光照图。所有组件均可在无任何人工标注的情况下进行训练。我们的方法无需参数,能够生成分辨率为512 x 512像素的高质量UV图,而以往方法通常生成256 x 256像素或更小。论文一经接收,我们将发布代码。

英文摘要:

Recent works based on convolutional encoder-decoder architecture and 3DMM parameterization have shown great potential for canonical view reconstruction from a single input image. Conventional CNN architectures benefit from exploiting the spatial correspondence between the input and output pixels. However, in 3D face reconstruction, the spatial misalignment between the input image (e.g. face) and the canonical/UV output makes the feature encoding-decoding process quite challenging. In this paper, to tackle this problem, we propose a new network architecture, namely the Affine Convolution Networks, which enables CNN based approaches to handle spatially non-corresponding input and output images and maintain high-fidelity quality output at the same time. In our method, an affine transformation matrix is learned from the affine convolution layer for each spatial location of the feature maps. In addition, we represent 3D human heads in UV space with multiple components, including diffuse maps for texture representation, position maps for geometry representation, and light maps for recovering more complex lighting conditions in the real world. All the components can be trained without any manual annotations. Our method is parametric-free and can generate high-quality UV maps at resolution of 512 x 512 pixels, while previous approaches normally generate 256 x 256 pixels or smaller. Our code will be released once the paper got accepted.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑