使用前视嵌入的高效条件人脸动画
Efficient conditioned face animation using frontally-viewed embedding
摘要：
随着来自地标的Few Shot面部动画质量的提高，新的应用成为可能，例如具有高度真实感的超低带宽视频聊天压缩。然而，为了改善现实世界条件下的体验，还有一些重要的挑战需要解决。特别是，目前的方法无法表示没有失真的剖面图，同时运行在低计算制度。我们专注于这个关键问题，通过引入一个多帧嵌入称为Frontalizer，以提高剖面视图渲染。除了这一核心改进之外，我们还探索了潜在代码调节代的学习以及地标，以更好地传达面部表情。我们的密集模型在包含头部运动的DFDC视频子集上实现了22%的感知质量改善和73%的地标误差减少。随着移动架构的下降，我们的模型表现优于以前的最先进水平（在两个数据集上将感知质量提高了16%以上，并将地标误差降低了47%以上），同时在iPhone 8上以非常低的带宽要求实时的运行。
英文摘要：
As the quality of few shot facial animation from landmarks increases, new applications become possible, such as ultra low bandwidth video chat compression with a high degree of realism. However, there are some important challenges to tackle in order to improve the experience in real world conditions. In particular, the current approaches fail to represent profile views without distortions, while running in a low compute regime. We focus on this key problem by introducing a multi-frames embedding dubbed Frontalizer to improve profile views rendering. In addition to this core improvement, we explore the learning of a latent code conditioning generations along with landmarks to better convey facial expressions. Our dense models achieves 22% of improvement in perceptual quality and 73% reduction of landmark error over the first order model baseline on a subset of DFDC videos containing head movements. Declined with mobile architectures, our models outperform the previous state-of-the-art (improving perceptual quality by more than 16% and reducing landmark error by more than 47% on two datasets) while running on real time on iPhone 8 with very low bandwidth requirements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
