从模糊到可信：利用3D生成先验增强低质量的对话头
From Blurry to Believable: Enhancing Low-quality Talking Heads with 3D Generative Priors
AI总结：
SuperHead通过动态感知3D反向过程，利用预训练3D生成模型先验知识，提升低质量对话头的3D重建与动画质量。
摘要：
创建高保真、可动画化的3D对话头对于沉浸式应用至关重要，但往往受到低质量图像或视频源的阻碍，这些源会产生较差的3D重建。在本文中，我们介绍了一个名为SuperHead的新框架，用于增强低分辨率、可动画的3D头身avata。核心挑战在于合成高质量的几何和纹理，同时在动画过程中确保3D和时间一致性，并保持主体身份。尽管在图像、视频和基于3D的超分辨率（SR）方面取得了进展，但现有SR技术无法处理动态3D输入。为了解决这个问题，SuperHead利用预训练3D生成模型的丰富先验知识，通过一种新的动态感知3D反向过程。此过程优化生成模型的潜在表示，以生成超分辨率的3D高斯点散（3DGS）头模型，然后将其连接到底层参数化头模型（例如FLAME）以进行动画。反向过程通过稀疏收集的放大2D面部渲染和相应的深度图进行联合监督，这些图是从多样的面部表情和相机视角捕获的，以确保在动态面部运动下的真实性。实验表明，SuperHead在动态运动下生成具有细粒度面部细节的avata，显著优于基线方法在视觉质量上。
英文摘要：
Creating high-fidelity, animatable 3D talking heads is crucial for immersive applications, yet often hindered by the prevalence of low-quality image or video sources, which yield poor 3D reconstructions. In this paper, we introduce SuperHead, a novel framework for enhancing low-resolution, animatable 3D head avatars. The core challenge lies in synthesizing high-quality geometry and textures, while ensuring both 3D and temporal consistency during animation and preserving subject identity. Despite recent progress in image, video and 3D-based super-resolution (SR), existing SR techniques are ill-equipped to handle dynamic 3D inputs. To address this, SuperHead leverages the rich priors from pre-trained 3D generative models via a novel dynamics-aware 3D inversion scheme. This process optimizes the latent representation of the generative model to produce a super-resolved 3D Gaussian Splatting (3DGS) head model, which is subsequently rigged to an underlying parametric head model (e.g., FLAME) for animation. The inversion is jointly supervised using a sparse collection of upscaled 2D face renderings and corresponding depth maps, captured from diverse facial expressions and camera viewpoints, to ensure realism under dynamic facial motions. Experiments demonstrate that SuperHead generates avatars with fine-grained facial details under dynamic motions, significantly outperforming baseline methods in visual quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
