基于眨眼嵌入和哈希网格地标编码的音频驱动说话人脸生成
Audio-Driven Talking Face Generation with Blink Embedding and Hash Grid Landmarks Encoding
AI总结：
本文提出基于眨眼嵌入和哈希网格地标编码的音频驱动说话人脸生成方法，通过动态地标变换器和神经辐射场提升生成质量。
摘要：
动态神经辐射场（NeRF）在生成高保真的说话肖像3D模型方面已显示出显著的成功。尽管在渲染速度和生成质量上取得了显著进展，但在准确且高效地捕捉说话肖像中的口部运动方面仍存在挑战。为了解决这一挑战，我们提出了一种基于眨眼嵌入和哈希网格地标编码的自动方法。该方法可以显著提高说话人脸的保真度。具体而言，我们利用以条件特征编码的面部特征，并通过动态地标变换器将音频特征作为残差项整合到我们的模型中。此外，我们采用神经辐射场来建模整个面部，从而获得逼真的面部表示。实验评估已验证了我们方法相对于现有方法的优越性。
英文摘要：
Dynamic Neural Radiance Fields (NeRF) have demonstrated considerable success in generating high-fidelity 3D models of talking portraits. Despite significant advancements in the rendering speed and generation quality, challenges persist in accurately and efficiently capturing mouth movements in talking portraits. To tackle this challenge, we propose an automatic method based on blink embedding and hash grid landmarks encoding in this study, which can substantially enhance the fidelity of talking faces. Specifically, we leverage facial features encoded as conditional features and integrate audio features as residual terms into our model through a Dynamic Landmark Transformer. Furthermore, we employ neural radiance fields to model the entire face, resulting in a lifelike face representation. Experimental evaluations have validated the superiority of our approach to existing methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
