BakedAvatar：烘焙神经场以实现实时头部化身合成
BakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesis
AI总结：
BakedAvatar提出一种将神经场烘焙为多层网格与静态纹理的实时头部化身合成方法，通过三阶段管线在保持高质量的同时大幅降低推理时间，支持交互式视图合成与表情姿态编辑。
摘要：
从视频中合成逼真的4D人类头部化身对于VR/AR、远程呈现和视频游戏应用至关重要。尽管现有的基于神经辐射场（NeRF）的方法能够实现高保真结果，但其计算开销限制了它们在实时应用中的使用。为了克服这一限制，我们提出了BakedAvatar，这是一种用于实时神经头部化身合成的新颖表示，可部署在标准多边形光栅化管线中。我们的方法从学习到的头部等值面中提取可变形多层网格，并计算与表情、姿态和视角相关的外观，这些外观可以被烘焙到静态纹理中，以实现高效的光栅化。因此，我们提出了一种用于神经头部化身合成的三阶段管线，包括学习连续变形、流形和辐射场，提取分层网格和纹理，以及利用差分光栅化微调纹理细节。实验结果表明，我们的表示生成的合成结果在质量上与其他最先进方法相当，同时显著减少了所需的推理时间。我们进一步展示了来自单目视频的各种头部化身合成结果，包括视图合成、面部重演、表情编辑和姿态编辑，所有这些都在交互式帧率下运行。
英文摘要：
Synthesizing photorealistic 4D human head avatars from videos is essential for VR/AR, telepresence, and video game applications. Although existing Neural Radiance Fields (NeRF)-based methods achieve high-fidelity results, the computational expense limits their use in real-time applications. To overcome this limitation, we introduce BakedAvatar, a novel representation for real-time neural head avatar synthesis, deployable in a standard polygon rasterization pipeline. Our approach extracts deformable multi-layer meshes from learned isosurfaces of the head and computes expression-, pose-, and view-dependent appearances that can be baked into static textures for efficient rasterization. We thus propose a three-stage pipeline for neural head avatar synthesis, which includes learning continuous deformation, manifold, and radiance fields, extracting layered meshes and textures, and fine-tuning texture details with differential rasterization. Experimental results demonstrate that our representation generates synthesis results of comparable quality to other state-of-the-art methods while significantly reducing the inference time required. We further showcase various head avatar synthesis results from monocular videos, including view synthesis, face reenactment, expression editing, and pose editing, all at interactive frame rates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
