基于合成先验的少样本可驱动头部头像反演
Synthetic Prior for Few-Shot Drivable Head Avatar Inversion
AI总结：
提出 SynShot 方法，利用大规模合成数据学习先验，结合 3D Gaussian splatting 和卷积编解码器，通过少样本微调实现可泛化的逼真可驱动头部头像反演。
摘要：
我们提出了 SynShot，一种基于合成先验的少样本可驱动头部头像反演新方法。我们解决了三个主要挑战。首先，训练可控的 3D 生成网络需要大量多样化的序列，而图像与高质量跟踪网格的配对并不总是可用。其次，真实数据的使用受到严格监管（例如在《通用数据保护条例》下，要求频繁删除模型和数据以应对参与者撤回同意的情况）。不受这些约束的合成数据成为一种极具吸引力的替代方案。第三，最先进的单目头像模型难以泛化到新视角和表情，缺乏强先验且常过拟合于特定视角分布。受仅在合成数据上训练的机器学习模型启发，我们提出了一种从具有多样身份、表情和视角的合成头部大型数据集中学习先验模型的方法。在少量输入图像下，SynShot 微调预训练的合成先验以弥合域差距，建模出能泛化至新表情和视角的逼真头部头像。我们使用 3D Gaussian splatting 和在 UV 纹理空间输出 Gaussian 参数的卷积编码器-解码器对头部头像进行建模。为应对头部不同部分（如皮肤与头发）的建模复杂度差异，我们在先验中嵌入了用于上采样各部分基元数量的显式控制。与 SOTA 单目及基于 GAN 的方法相比，SynShot 显著提升了新视角与表情的合成效果。
英文摘要：
We present SynShot, a novel method for the few-shot inversion of a drivable head avatar based on a synthetic prior. We tackle three major challenges. First, training a controllable 3D generative network requires a large number of diverse sequences, for which pairs of images and high-quality tracked meshes are not always available. Second, the use of real data is strictly regulated (e.g., under the General Data Protection Regulation, which mandates frequent deletion of models and data to accommodate a situation when a participant's consent is withdrawn). Synthetic data, free from these constraints, is an appealing alternative. Third, state-of-the-art monocular avatar models struggle to generalize to new views and expressions, lacking a strong prior and often overfitting to a specific viewpoint distribution. Inspired by machine learning models trained solely on synthetic data, we propose a method that learns a prior model from a large dataset of synthetic heads with diverse identities, expressions, and viewpoints. With few input images, SynShot fine-tunes the pretrained synthetic prior to bridge the domain gap, modeling a photorealistic head avatar that generalizes to novel expressions and viewpoints. We model the head avatar using 3D Gaussian splatting and a convolutional encoder-decoder that outputs Gaussian parameters in UV texture space. To account for the different modeling complexities over parts of the head (e.g., skin vs hair), we embed the prior with explicit control for upsampling the number of per-part primitives. Compared to SOTA monocular and GAN-based methods, SynShot significantly improves novel view and expression synthesis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
