arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.04558cs.CV

MonoGaussianAvatar:基于单目高斯点的头部头像

MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar

Tsinghua University(清华大学);Communication University of China(中国传媒大学);Harbin Institute of Technology(哈尔滨工业大学)

Yufan Chen, Lizhen Wang, Qijing Li, Hongjiang Xiao, Shengping Zhang, Hongxun Yao, Yebin Liu

AI总结:

本文提出MonoGaussianAvatar,用3D高斯点和高斯形变场从单目视频重建可动画头部头像,兼顾灵活拓扑、高效形变与渲染,并达到最先进性能。

摘要:

从单目肖像视频序列重建照片级真实头部头像并对其进行动画化的能力,是弥合虚拟世界与现实世界之间差距的关键一步。近期头部头像技术的进展,包括显式3D可变形网格(3DMM)、点云和神经隐式表示,已被用于这一持续开展的研究。然而,基于3DMM的方法受限于其固定拓扑,基于点的方法因涉及数量庞大的点而承受沉重的训练负担,而最后一类方法则在形变灵活性和渲染效率方面存在局限。针对这些挑战,我们提出MonoGaussianAvatar(基于单目高斯点的头部头像),这是一种利用3D高斯点表示并结合高斯形变场,从单目肖像视频中学习显式头部头像的新颖方法。我们用具有可适应形状的高斯点定义头部头像,从而实现灵活拓扑。这些点通过高斯形变场运动,以与人物的目标姿态和表情保持一致,从而促进高效形变。此外,高斯点具有可控的形状、大小、颜色和不透明度,并结合高斯泼溅(Gaussian splatting),可实现高效训练和渲染。实验证明了我们方法的优越性能,其在以往方法中取得了最先进的结果。

英文摘要:

The ability to animate photo-realistic head avatars reconstructed from monocular portrait video sequences represents a crucial step in bridging the gap between the virtual and real worlds. Recent advancements in head avatar techniques, including explicit 3D morphable meshes (3DMM), point clouds, and neural implicit representation have been exploited for this ongoing research. However, 3DMM-based methods are constrained by their fixed topologies, point-based approaches suffer from a heavy training burden due to the extensive quantity of points involved, and the last ones suffer from limitations in deformation flexibility and rendering efficiency. In response to these challenges, we propose MonoGaussianAvatar (Monocular Gaussian Point-based Head Avatar), a novel approach that harnesses 3D Gaussian point representation coupled with a Gaussian deformation field to learn explicit head avatars from monocular portrait videos. We define our head avatars with Gaussian points characterized by adaptable shapes, enabling flexible topology. These points exhibit movement with a Gaussian deformation field in alignment with the target pose and expression of a person, facilitating efficient deformation. Additionally, the Gaussian points have controllable shape, size, color, and opacity combined with Gaussian splatting, allowing for efficient training and rendering. Experiments demonstrate the superior performance of our method, which achieves state-of-the-art results among previous methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑