3DFacePolicy：基于动作控制的音频驱动3D面部动画
3DFacePolicy: Audio-Driven 3D Facial Animation Based on Action Control
AI总结：
提出3DFacePolicy，通过引入“动作”概念将逐帧顶点生成转化为基于动作的控制范式，利用diffusion policy预测动作序列，在多个数据集上显著优于现有方法。
摘要：
音频驱动的3D面部动画在研究和应用中均取得了显著进展。由于最近的基线方法采用逐帧顶点生成方法，难以生成自然连续的面部运动，我们提出了3DFacePolicy，这是一项先驱工作，通过“动作”的概念引入了跨连续帧顶点轨迹变化的新颖定义。通过预测每个顶点编码帧间运动的动作序列，我们将顶点生成方法重新表述为基于动作的控制范式。具体而言，我们利用机器人控制机制diffusion policy，预测以音频和顶点状态为条件的动作序列。在VOCASET和BIWI数据集上的大量实验表明，我们的方法显著优于最先进的方法，并且特别擅长生成动态、富有表现力且自然平滑的面部动画。
英文摘要：
Audio-driven 3D facial animation has achieved significant progress in both research and applications. While recent baselines struggle to generate natural and continuous facial movements due to their frame-by-frame vertex generation approach, we propose 3DFacePolicy, a pioneer work that introduces a novel definition of vertex trajectory changes across consecutive frames through the concept of "action". By predicting action sequences for each vertex that encode frame-to-frame movements, we reformulate vertex generation approach into an action-based control paradigm. Specifically, we leverage a robotic control mechanism, diffusion policy, to predict action sequences conditioned on both audio and vertex states. Extensive experiments on VOCASET and BIWI datasets demonstrate that our approach significantly outperforms state-of-the-art methods and is particularly expert in dynamic, expressive and naturally smooth facial animations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
