3DiFACE：合成与编辑整体 3D 面部动画
3DiFACE: Synthesizing and Editing Holistic 3D Facial Animation
AI总结：
本文提出 3DiFACE，利用全卷积扩散、说话风格个性化与稀疏引导运动扩散，从单一音频生成并可关键帧编辑多样且真实的整体 3D 面部动画。
摘要：
对于当前由语音驱动的 3D 面部动画方法而言，创建具有精确控制和真实头部运动的个性化 3D 动画仍然具有挑战性。编辑这些动画尤其复杂且耗时，需要精确控制，并且通常由技能娴熟的动画师处理。大多数现有工作专注于控制合成动画的风格或情感，无法编辑或重新生成输入动画的局部。它们还忽视了一个事实：同一段音频输入可以对应多种合理的唇部和头部运动。为应对这些挑战，我们提出 3DiFACE，这是一种用于整体式语音驱动 3D 面部动画的新颖方法。我们的方法能够针对单一音频输入生成多样且合理的唇部与头部运动，并允许通过关键帧设置和插值进行编辑。具体而言，我们提出一种全卷积扩散模型，能够利用训练语料中视位（viseme）层面的多样性。此外，我们采用说话风格个性化以及新颖的稀疏引导运动扩散，以实现精确控制和编辑。通过定量和定性评估，我们证明本方法能够在给定单一音频输入的情况下生成并编辑多样的整体 3D 面部动画，并在高保真度与多样性之间进行控制。代码和模型可在此处获取：https://balamuruganthambiraja.github.io/3DiFACE
英文摘要：
Creating personalized 3D animations with precise control and realistic head motions remains challenging for current speech-driven 3D facial animation methods. Editing these animations is especially complex and time consuming, requires precise control and typically handled by highly skilled animators. Most existing works focus on controlling style or emotion of the synthesized animation and cannot edit/regenerate parts of an input animation. They also overlook the fact that multiple plausible lip and head movements can match the same audio input. To address these challenges, we present 3DiFACE, a novel method for holistic speech-driven 3D facial animation. Our approach produces diverse plausible lip and head motions for a single audio input and allows for editing via keyframing and interpolation. Specifically, we propose a fully-convolutional diffusion model that can leverage the viseme-level diversity in our training corpus. Additionally, we employ a speaking-style personalization and a novel sparsely-guided motion diffusion to enable precise control and editing. Through quantitative and qualitative evaluations, we demonstrate that our method is capable of generating and editing diverse holistic 3D facial animations given a single audio input, with control between high fidelity and diversity. Code and models are available here: https://balamuruganthambiraja.github.io/3DiFACE
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
