3DiFACE：基于扩散模型的语音驱动3D面部动画与编辑
3DiFACE: Diffusion-based Speech-driven 3D Facial Animation and Editing
AI总结：
3DiFACE提出一种轻量级音频条件扩散模型，解决语音驱动3D面部动画的一对多映射问题，支持随机生成与运动编辑，仅需短视频即可个性化微调，在保真度和多样性上超越现有方法。
摘要：
我们提出了3DiFACE，一种用于个性化语音驱动3D面部动画与编辑的新方法。现有方法从语音中确定性地预测面部动画，但忽略了语音与面部表情之间固有的一对多关系，即存在多个与音频输入匹配的合理面部表情动画。在内容创作中，能够修改生成的运动或指定关键帧尤为重要。为了实现随机性以及运动编辑，我们提出了一种轻量级的音频条件扩散模型用于3D面部运动。该扩散模型可以在小型3D运动数据集上训练，同时保持富有表现力的唇部运动输出。此外，它可以针对特定对象进行微调，仅需该人物的短视频即可。通过定量和定性评估，我们表明我们的方法优于现有的最先进技术，并产生具有更高保真度和多样性的语音驱动动画。
英文摘要：
We present 3DiFACE, a novel method for personalized speech-driven 3D facial animation and editing. While existing methods deterministically predict facial animations from speech, they overlook the inherent one-to-many relationship between speech and facial expressions, i.e., there are multiple reasonable facial expression animations matching an audio input. It is especially important in content creation to be able to modify generated motion or to specify keyframes. To enable stochasticity as well as motion editing, we propose a lightweight audio-conditioned diffusion model for 3D facial motion. This diffusion model can be trained on a small 3D motion dataset, maintaining expressive lip motion output. In addition, it can be finetuned for specific subjects, requiring only a short video of the person. Through quantitative and qualitative evaluations, we show that our method outperforms existing state-of-the-art techniques and yields speech-driven animations with greater fidelity and diversity.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
