GAF：通过多视图扩散从单目视频重建高斯化身
GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion
AI总结：
针对单目视频3D头部化身重建中观测有限导致的伪影问题，提出GAF方法，结合多视图扩散模型先验和FLAME法线贴图偏置，提升高斯溅射渲染的视角一致性与保真度，在NeRSemble数据集上优于现有方法。
摘要：
我们提出了一种从智能手机等消费级设备拍摄的单目视频中重建可动画3D高斯化身的新方法。由于观测有限，此类记录的真实感3D头部化身重建具有挑战性，未观测区域约束不足，可能导致新视角出现伪影。为解决该问题，我们引入多视图头部扩散模型，利用其先验填充缺失区域，确保高斯溅射渲染中的视角一致性。为实现精确视角控制，我们使用基于FLAME的头部重建渲染的法线贴图，提供像素对齐的归纳偏置。我们还将扩散模型以从输入图像提取的VAE特征为条件，以保留面部身份和外观细节。对于高斯化身重建，我们通过使用迭代去噪图像作为伪真值来提取多视图扩散先验，有效缓解过饱和问题。我们在NeRSemble数据集上评估了方法，表明GAF在新视角合成方面优于先前的最先进方法，并且展示了从消费级设备拍摄的单目视频中更高保真度的化身重建。
英文摘要：
We propose a novel approach for reconstructing animatable 3D Gaussian avatars from monocular videos captured by commodity devices like smartphones. Photorealistic 3D head avatar reconstruction from such recordings is challenging due to limited observations, which leaves unobserved regions under-constrained and can lead to artifacts in novel views. To address this problem, we introduce a multi-view head diffusion model, leveraging its priors to fill in missing regions and ensure view consistency in Gaussian splatting renderings. To enable precise viewpoint control, we use normal maps rendered from FLAME-based head reconstruction, which provides pixel-aligned inductive biases. We also condition the diffusion model on VAE features extracted from the input image to preserve facial identity and appearance details. For Gaussian avatar reconstruction, we distill multi-view diffusion priors by using iteratively denoised images as pseudo-ground truths, effectively mitigating over-saturation issues. To further improve photorealism, we apply latent upsampling priors to refine the denoised latent before decoding it into an image. We evaluate our method on the NeRSemble dataset, showing that GAF outperforms previous state-of-the-art methods in novel view synthesis. Furthermore, we demonstrate higher-fidelity avatar reconstructions from monocular videos captured on commodity devices.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
