GaussianEmoTalker：基于音频驱动和混合形状的3D高斯泼溅实时情感说话头合成
GaussianEmoTalker: Real-Time Emotional Talking Head Synthesis with Audio-Driven and Blendshape-Based 3D Gaussian Splatting
AI总结：
提出GaussianEmoTalker框架，利用3D高斯泼溅和混合形状，通过中性到情感残差变形实现实时、可控强度的情感说话头合成，兼顾唇同步与渲染质量。
摘要：
音频驱动的说话头合成在唇同步和视觉质量方面取得了显著进展，但生成具有可控强度的表情化虚拟形象仍具挑战性，尤其是在实时约束下。本文提出GaussianEmoTalker，一种基于3D高斯泼溅的音频驱动实时情感说话头合成框架。我们不直接从语音预测最终情感虚拟形象，而是将情感动画形式化为中性到情感的残差变形问题。GaussianEmoTalker首先利用高斯混合形状构建身份特定的中性说话空间，提供高保真高斯属性和音素同步的中性运动。然后，通过结合网格位移线索、音频特征、情感类别和强度编码，预测情感条件残差变形。为融合这些异质信号，我们引入空间-音频-情感注意力模块，估计高斯属性的偏移量，以实现富有表现力和时间稳定的渲染。大量实验表明，与近期情感说话头方法相比，GaussianEmoTalker在视频质量、精确唇同步、可控情感表达和实时渲染方面具有竞争力。我们的项目页面见该URL。
英文摘要：
Audio-driven talking head synthesis has achieved impressive progress in lip synchronization and visual quality, yet generating expressive emotional avatars with controllable intensity remains challenging, especially under real-time constraints. In this paper, we present GaussianEmoTalker, an audio-driven framework for real-time emotional talking head synthesis based on 3D Gaussian Splatting. Instead of directly predicting the final emotional avatar from speech, we formulate emotional animation as a neutral-to-emotional residual deformation problem. GaussianEmoTalker first constructs an identity-specific neutral talking space with GaussianBlendshapes, which provides high-fidelity Gaussian attributes and phoneme-synchronized neutral motion. It then predicts an emotion-conditioned residual deformation by combining mesh displacement cues, audio features, emotion categories, and intensity encodings. To fuse these heterogeneous signals, we introduce a spatial-audio-emotion attention module that estimates the offsets of Gaussian attributes for expressive and temporally stable rendering. Extensive experiments demonstrate that GaussianEmoTalker achieves competitive video quality, accurate lip synchronization, controllable emotional expression, and real-time rendering compared with recent emotional talking head methods. Our project page is available at https://njust-yang.github.io/GaussianEmoTalker.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
