HyperGaussians: 高维高斯点云用于高质量可动画人脸仿生体
HyperGaussians: High-Dimensional Gaussian Splatting for High-Fidelity Animatable Face Avatars
AI总结：
本文提出HyperGaussians，一种高维多变量高斯点云的扩展，用于生成高质量可动画人脸仿生体。通过引入可学习的局部嵌入条件，提升表达能力，解决传统3DGS在非线性变形和细节表现上的不足。
摘要：
我们介绍了HyperGaussians，一种3D高斯点云的新扩展，用于生成高质量可动画人脸仿生体。从视频中创建如此细致的人脸仿生体是一项具有挑战性的问题，广泛应用于增强现实和虚拟现实。尽管静态人脸取得了巨大成功，但单目视频中可动画仿生体仍处于怪诞谷。事实上的标准，3D高斯点云（3DGS），通过一组3D高斯原始体表示人脸。3DGS在渲染静态人脸方面表现出色，但最先进的方法仍难以处理非线性变形、复杂光照效果和精细细节。尽管大多数相关工作专注于从表情代码预测更好的高斯参数，我们重新思考了3D高斯表示本身以及如何使其更具表现力。我们的洞察促使我们提出一种新的3D高斯扩展，即高维多变量高斯，称为'HyperGaussians'。更高的维度通过在可学习的局部嵌入上进行条件化来增加表达能力。然而，点云HyperGaussians计算上很昂贵，因为它需要求解高维协方差矩阵的逆。我们通过重新参数化协方差矩阵，称为'逆协方差技巧'来解决这个问题。这种技巧提高了效率，使HyperGaussians能够无缝集成到现有模型中。为了证明这一点，我们将HyperGaussians插入到最先进的快速单目人脸仿生体模型中：FlashAvatar。在19个受试者来自4个面部数据集的评估中，HyperGaussians在数值和视觉上均优于3DGS，特别是在高频细节如眼镜框、牙齿、复杂面部运动和镜面反射方面。
英文摘要：
We introduce HyperGaussians, a novel extension of 3D Gaussian Splatting for high-quality animatable face avatars. Creating such detailed face avatars from videos is a challenging problem and has numerous applications in augmented and virtual reality. While tremendous successes have been achieved for static faces, animatable avatars from monocular videos still fall in the uncanny valley. The de facto standard, 3D Gaussian Splatting (3DGS), represents a face through a collection of 3D Gaussian primitives. 3DGS excels at rendering static faces, but the state-of-the-art still struggles with nonlinear deformations, complex lighting effects, and fine details. While most related works focus on predicting better Gaussian parameters from expression codes, we rethink the 3D Gaussian representation itself and how to make it more expressive. Our insights lead to a novel extension of 3D Gaussians to high-dimensional multivariate Gaussians, dubbed 'HyperGaussians'. The higher dimensionality increases expressivity through conditioning on a learnable local embedding. However, splatting HyperGaussians is computationally expensive because it requires inverting a high-dimensional covariance matrix. We solve this by reparameterizing the covariance matrix, dubbed the 'inverse covariance trick'. This trick boosts the efficiency so that HyperGaussians can be seamlessly integrated into existing models. To demonstrate this, we plug in HyperGaussians into the state-of-the-art in fast monocular face avatars: FlashAvatar. Our evaluation on 19 subjects from 4 face datasets shows that HyperGaussians outperform 3DGS numerically and visually, particularly for high-frequency details like eyeglass frames, teeth, complex facial movements, and specular reflections.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
