NLDF：用于高效3D说话人生成的神经光动态场
NLDF: Neural Light Dynamic Fields for Efficient 3D Talking Head Generation
AI总结：
针对NeRF说话人生成渲染慢的问题，提出基于光段与知识蒸馏的NLDF模型，并用主动池训练聚焦嘴眉高频运动，实现约30倍提速且质量相当。
摘要：
基于神经辐射场模型的说话人头部生成已展现出良好的视觉效果。然而，NeRF缓慢的渲染速度严重限制了其应用，原因在于合成一个像素需要对数百个采样点进行繁重的计算。本文提出一种新颖的神经光动态场模型，旨在实现高质量3D说话人脸生成并显著提速。NLDF基于光段表示光场，并使用深度网络一次性学习整条光束的信息。在学习过程中采用知识蒸馏，利用基于NeRF的合成结果指导NLDF中光段的正确着色。此外，提出一种新颖的主动池训练策略，以聚焦高频运动，特别是说话者的嘴部和眉毛。所提方法有效表示了3D说话视频生成中的面部光动态，并且与最先进的基于NeRF的方法相比，速度提升约30倍，同时生成视觉质量相当。
英文摘要：
Talking head generation based on the neural radiation fields model has shown promising visual effects. However, the slow rendering speed of NeRF seriously limits its application, due to the burdensome calculation process over hundreds of sampled points to synthesize one pixel. In this work, a novel Neural Light Dynamic Fields model is proposed aiming to achieve generating high quality 3D talking face with significant speedup. The NLDF represents light fields based on light segments, and a deep network is used to learn the entire light beam's information at once. In learning the knowledge distillation is applied and the NeRF based synthesized result is used to guide the correct coloration of light segments in NLDF. Furthermore, a novel active pool training strategy is proposed to focus on high frequency movements, particularly on the speaker mouth and eyebrows. The propose method effectively represents the facial light dynamics in 3D talking video generation, and it achieves approximately 30 times faster speed compared to state of the art NeRF based method, with comparable generation visual quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
