解耦身份，协同情感：相关性感知的情感说话肖像生成
Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation
AI总结：
针对现有情感说话头生成中情感利用不足、身份泄漏和相关性孤立问题，提出DICE-Talk框架，通过解耦情感嵌入器、相关性增强条件模块和情感判别目标，在MEAD和HDTF上实现更优的情感准确性和身份保持。
摘要：
近年来，说话头生成（THG）的进展通过扩散模型实现了令人印象深刻的唇部同步和视觉质量；然而，现有方法在生成情感表达丰富的肖像时，难以保持说话者身份。我们识别了当前情感说话头生成中的三个关键限制：对音频固有情感线索的利用不足、情感表示中的身份泄漏以及情感相关性的孤立学习。为解决这些挑战，我们提出了一种名为DICE-Talk的新框架，遵循将身份与情感解耦，然后协同具有相似特征的情感的思路。首先，我们开发了一个解耦的情感嵌入器，通过跨模态注意力联合建模视听情感线索，将情感表示为与身份无关的高斯分布。其次，我们引入了一个带有可学习情感库的相关性增强情感条件模块，通过向量量化和基于注意力的特征聚合显式捕获情感间关系。第三，我们设计了一个情感判别目标，通过潜在空间分类在扩散过程中强制情感一致性。在MEAD和HDTF数据集上的大量实验证明了我们方法的优越性，在情感准确性上超越了最先进的方法，同时保持了有竞争力的唇部同步性能。定性结果和用户研究进一步证实了我们的方法能够生成具有丰富、相关情感表达的保持身份的肖像，并能自然地适应未见过的身份。
英文摘要：
Recent advances in Talking Head Generation (THG) have achieved impressive lip synchronization and visual quality through diffusion models; yet existing methods struggle to generate emotionally expressive portraits while preserving speaker identity. We identify three critical limitations in current emotional talking head generation: insufficient utilization of audio's inherent emotional cues, identity leakage in emotion representations, and isolated learning of emotion correlations. To address these challenges, we propose a novel framework dubbed as DICE-Talk, following the idea of disentangling identity with emotion, and then cooperating emotions with similar characteristics. First, we develop a disentangled emotion embedder that jointly models audio-visual emotional cues through cross-modal attention, representing emotions as identity-agnostic Gaussian distributions. Second, we introduce a correlation-enhanced emotion conditioning module with learnable Emotion Banks that explicitly capture inter-emotion relationships through vector quantization and attention-based feature aggregation. Third, we design an emotion discrimination objective that enforces affective consistency during the diffusion process through latent-space classification. Extensive experiments on MEAD and HDTF datasets demonstrate our method's superiority, outperforming state-of-the-art approaches in emotion accuracy while maintaining competitive lip-sync performance. Qualitative results and user studies further confirm our method's ability to generate identity-preserving portraits with rich, correlated emotional expressions that naturally adapt to unseen identities.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
