arXiv 2408.060192024-08cs.CV

HeadGAP：基于可泛化高斯先验的少样本3D头部化身

HeadGAP: Few-Shot 3D Head Avatar via Generalizable Gaussian Priors

ByteDance（字节跳动）；ShanghaiTech University（上海科技大学）

Xiaozheng Zheng, Chao Wen, Zhaohu Li, Weiyi Zhang, Zhuo Su, Xu Chang, Yang Zhao, Zheng Lv, Xiaoyuan Zhang, Yongjie Zhang, Guidong Wang, Lan Xu

备注：: Accepted to 3DV 2025. Project page: https://headgap.github.io/

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对少样本3D头部化身构建的欠约束问题，提出HeadGAP框架，通过基于高斯溅射的自解码器网络从大规模动态数据学习头部先验，结合反演微调实现高保真、可动画的少样本个性化头部生成。

摘要：

本文提出了一种新颖的3D头部化身构建方法，能够从少样本真实场景数据中实现高保真、可动画化的鲁棒泛化。鉴于该问题的欠约束特性，引入先验知识至关重要。为此，我们提出一个包含先验学习阶段和化身构建阶段的框架。先验学习阶段利用从大规模多视图动态数据集中提取的3D头部先验，化身构建阶段则将这些先验应用于少样本个性化。我们的方法通过采用基于Gaussian Splatting（高斯溅射）的自解码器网络结合基于部件的动态建模，有效捕获这些先验。该方法使用身份共享编码搭配针对个体身份的个性化潜在编码，来学习高斯基元的属性。在化身构建阶段，我们通过反演和微调策略实现快速的头部化身个性化。大量实验表明，我们的模型有效利用了头部先验，并成功将其泛化到少样本个性化任务中，实现了照片级真实感的渲染质量、多视图一致性和稳定的动画效果。

英文摘要：

In this paper, we present a novel 3D head avatar creation approach capable of generalizing from few-shot in-the-wild data with high-fidelity and animatable robustness. Given the underconstrained nature of this problem, incorporating prior knowledge is essential. Therefore, we propose a framework comprising prior learning and avatar creation phases. The prior learning phase leverages 3D head priors derived from a large-scale multi-view dynamic dataset, and the avatar creation phase applies these priors for few-shot personalization. Our approach effectively captures these priors by utilizing a Gaussian Splatting-based auto-decoder network with part-based dynamic modeling. Our method employs identity-shared encoding with personalized latent codes for individual identities to learn the attributes of Gaussian primitives. During the avatar creation phase, we achieve fast head avatar personalization by leveraging inversion and fine-tuning strategies. Extensive experiments demonstrate that our model effectively exploits head priors and successfully generalizes them to few-shot personalization, achieving photo-realistic rendering quality, multi-view consistency, and stable animation.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文