通过统一多模态空间中的扩散实现可控的表现力3D面部动画
Controllable Expressive 3D Facial Animation via Diffusion in a Unified Multimodal Space
AI总结：
针对音频驱动3D面部动画的单模态控制局限与确定性映射表现力不足问题，提出基于扩散的框架，通过多模态情感绑定与带注意力的潜在扩散模型，实现灵活可控且富有表现力的动画生成，性能优于现有方法。
摘要：
音频驱动的情感化3D面部动画面临两大挑战：（1）依赖单模态控制信号（视频、文本或情感标签），未利用其互补优势实现全面的情感操控；（2）基于确定性回归的映射限制了情感表达与非言语行为的随机性，制约了合成动画的表现力。为解决这些问题，我们提出一种基于扩散的可控表现力3D面部动画框架。该方法有两项核心创新：（1）以FLAME为中心的多模态情感绑定策略，通过对比学习对齐文本、音频、情感标签等多种模态，支持从多信号源进行灵活的情感控制；（2）带内容感知注意力与情感引导层的基于注意力的潜在扩散模型，在保持时间连贯性与自然面部动态的同时丰富了动作多样性。大量实验表明，我们的方法在多数指标上优于现有方法，情感相似度提升21.6%，同时保留了生理上合理的面部动态。项目页面：https://kangweiiliu.github.io/Control_3D_Animation。
英文摘要：
Audio-driven emotional 3D facial animation encounters two significant challenges: (1) reliance on single-modal control signals (videos, text, or emotion labels) without leveraging their complementary strengths for comprehensive emotion manipulation, and (2) deterministic regression-based mapping that constrains the stochastic nature of emotional expressions and non-verbal behaviors, limiting the expressiveness of synthesized animations. To address these challenges, we present a diffusion-based framework for controllable expressive 3D facial animation. Our approach introduces two key innovations: (1) a FLAME-centered multimodal emotion binding strategy that aligns diverse modalities (text, audio, and emotion labels) through contrastive learning, enabling flexible emotion control from multiple signal sources, and (2) an attention-based latent diffusion model with content-aware attention and emotion-guided layers, which enriches motion diversity while maintaining temporal coherence and natural facial dynamics. Extensive experiments demonstrate that our method outperforms existing approaches across most metrics, achieving a 21.6\% improvement in emotion similarity while preserving physiologically plausible facial dynamics. Project Page: https://kangweiiliu.github.io/Control_3D_Animation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
