FED-NeRF：在动态NeRF上实现高3D一致性和时间连贯性的人脸视频编辑
FED-NeRF: Achieve High 3D Consistency and Temporal Coherence for Face Video Editing on Dynamic NeRF
AI总结：
本文提出FED-NeRF，一种基于动态人脸GAN-NeRF的新型人脸视频编辑架构，通过恢复潜在编码和3D人脸几何实现多视角一致性，并利用稳定器保持连续帧表情平滑变化以解决抖动问题，实现时间连贯性。
摘要：
GAN-NeRF结构的成功使得在NeRF上进行人脸编辑能够保持3D视角一致性。然而，在编辑视频序列时，同时实现多视角一致性和时间连贯性仍然是一个艰巨的挑战。本文提出了一种基于动态人脸GAN-NeRF结构的新型人脸视频编辑架构，该架构有效地利用视频序列来恢复潜在编码和3D人脸几何结构。通过对潜在编码进行编辑，可以确保对人脸进行多视角一致的编辑，这一点已通过在动态NeRF中生成的编辑图像上进行多视角立体重建得到验证。由于人脸几何结构的估计是逐帧进行的，这可能会引入抖动问题。我们提出了一种稳定器，通过保持连续帧中人脸表情的平滑变化来维持时间连贯性。定量和定性分析表明，作为开创性的4D人脸视频编辑器，与独立处理身份和运动的现有基于2D或3D的方法相比，我们的方法取得了最先进的性能。代码将会发布。
英文摘要：
The success of the GAN-NeRF structure has enabled face editing on NeRF to maintain 3D view consistency. However, achieving simultaneously multi-view consistency and temporal coherence while editing video sequences remains a formidable challenge. This paper proposes a novel face video editing architecture built upon the dynamic face GAN-NeRF structure, which effectively utilizes video sequences to restore the latent code and 3D face geometry. By editing the latent code, multi-view consistent editing on the face can be ensured, as validated by multiview stereo reconstruction on the resulting edited images in our dynamic NeRF. As the estimation of face geometries occurs on a frame-by-frame basis, this may introduce a jittering issue. We propose a stabilizer that maintains temporal coherence by preserving smooth changes of face expressions in consecutive frames. Quantitative and qualitative analyses reveal that our method, as the pioneering 4D face video editor, achieves state-of-the-art performance in comparison to existing 2D or 3D-based approaches independently addressing identity and motion. Codes will be released.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
