真实表演驱动的风格感知面部动画的无监督学习
Unsupervised Learning of Style-Aware Facial Animation from Real Acting Performances
AI总结：
提出一种基于混合形状、动态纹理和神经渲染的文本/语音驱动面部动画方法，通过VAE和条件CNN无监督学习解耦表演风格，并用U-Net优化渲染，显著提升质量。
摘要：
本文提出了一种新颖的方法，用于基于混合形状几何、动态纹理和神经渲染的照片级逼真头部模型的文本/语音驱动动画。训练用于几何和纹理的VAE，得到一个参数化模型，能够从潜在特征向量中准确捕捉和逼真合成面部表情。我们的动画方法基于条件CNN，将文本或语音转换为动画参数序列。与先前方法相比，我们的动画模型以无监督方式学习解耦/合成不同的表演风格，仅需描述训练序列内容的语音标签。为了实现逼真的实时渲染，我们训练了一个U-Net，通过计算改进的像素颜色和前景遮罩来细化基于光栅化的渲染。我们在定性/定量上将我们的框架与最近的头部建模和面部动画方法进行比较，并通过用户研究评估感知的渲染/动画质量，结果表明与最先进的方法相比有大幅改进。
英文摘要：
This paper presents a novel approach for text/speech-driven animation of a photo-realistic head model based on blend-shape geometry, dynamic textures, and neural rendering. Training a VAE for geometry and texture yields a parametric model for accurate capturing and realistic synthesis of facial expressions from a latent feature vector. Our animation method is based on a conditional CNN that transforms text or speech into a sequence of animation parameters. In contrast to previous approaches, our animation model learns disentangling/synthesizing different acting-styles in an unsupervised manner, requiring only phonetic labels that describe the content of training sequences. For realistic real-time rendering, we train a U-Net that refines rasterization-based renderings by computing improved pixel colors and a foreground matte. We compare our framework qualitatively/quantitatively against recent methods for head modeling as well as facial animation and evaluate the perceived rendering/animation quality in a user-study, which indicates large improvements compared to state-of-the-art approaches
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
