用于动画化风格视频肖像的嵌入式表示学习网络
Embedded Representation Learning Network for Animating Styled Video Portrait
AI总结：
针对现有NeRF说话人合成中风格不可控和颈部伪影问题，提出两阶段嵌入式表示学习网络ERLNet，通过音频驱动FLAME模块和双分支融合NeRF实现更逼真的风格化说话人渲染。
摘要：
说话人头像生成因其广泛的应用前景，尤其是在数字虚拟形象和3D动画设计领域，近年来引起了相当大的关注。受这一实际需求的启发，一些工作探索了神经辐射场（NeRF）来合成说话人头像。然而，这些基于NeRF的方法面临两个挑战：（1）难以生成风格可控的说话人头像。（2）渲染图像中颈部周围存在位移伪影。为了克服这两个挑战，我们提出了一种新颖的生成范式——嵌入式表示学习网络（ERLNet），该网络包含两个学习阶段。首先，构建音频驱动FLAME（ADF）模块，以生成与内容音频和风格视频同步的面部表情和头部姿态序列。其次，给定由ADF推导出的序列，一种新颖的双分支融合NeRF（DBF-NeRF）探索这些内容以渲染最终图像。大量实证研究表明，这两个阶段的协作有效地使我们的方法能够渲染出比现有算法更逼真的说话人头像。
英文摘要：
The talking head generation recently attracted considerable attention due to its widespread application prospects, especially for digital avatars and 3D animation design. Inspired by this practical demand, several works explored Neural Radiance Fields (NeRF) to synthesize the talking heads. However, these methods based on NeRF face two challenges: (1) Difficulty in generating style-controllable talking heads. (2) Displacement artifacts around the neck in rendered images. To overcome these two challenges, we propose a novel generative paradigm \textit{Embedded Representation Learning Network} (ERLNet) with two learning stages. First, the \textit{ audio-driven FLAME} (ADF) module is constructed to produce facial expression and head pose sequences synchronized with content audio and style video. Second, given the sequence deduced by the ADF, one novel \textit{dual-branch fusion NeRF} (DBF-NeRF) explores these contents to render the final images. Extensive empirical studies demonstrate that the collaboration of these two stages effectively facilitates our method to render a more realistic talking head than the existing algorithms.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
