DisentTalk：基于语义解耦扩散模型的跨语言说话人脸生成
DisentTalk: Cross-lingual Talking Face Generation via Semantic Disentangled Diffusion Model
AI总结：
提出DisentTalk，通过语义解耦扩散模型分解3DMM参数，实现跨语言说话人脸生成，并构建CHDTF数据集，在唇同步、表情和时间一致性上超越现有方法。
摘要：
近年来，说话人脸生成技术的进展显著提升了面部动画合成的质量。然而，现有方法面临根本性限制：基于3DMM的方法保持了时间一致性，但缺乏细粒度的区域控制；而基于Stable Diffusion的方法支持空间操作，却存在时间不一致的问题。这些方法的整合受到不兼容的控制机制和面部表示语义纠缠的阻碍。本文提出DisentTalk，引入一种数据驱动的语义解耦框架，将3DMM表情参数分解为有意义的子空间，以实现细粒度的面部控制。基于这种解耦表示，我们开发了一种在3DMM参数空间中运行的分层潜在扩散架构，集成区域感知注意力机制，以确保空间精度和时间一致性。为解决高质量中文训练数据稀缺的问题，我们引入了CHDTF，一个中文高清说话人脸数据集。大量实验表明，在包括唇部同步、表情质量和时间一致性在内的多项指标上，该方法优于现有方法。项目页面：https://kangweiiliu.github.io/DisentTalk。
英文摘要：
Recent advances in talking face generation have significantly improved facial animation synthesis. However, existing approaches face fundamental limitations: 3DMM-based methods maintain temporal consistency but lack fine-grained regional control, while Stable Diffusion-based methods enable spatial manipulation but suffer from temporal inconsistencies. The integration of these approaches is hindered by incompatible control mechanisms and semantic entanglement of facial representations. This paper presents DisentTalk, introducing a data-driven semantic disentanglement framework that decomposes 3DMM expression parameters into meaningful subspaces for fine-grained facial control. Building upon this disentangled representation, we develop a hierarchical latent diffusion architecture that operates in 3DMM parameter space, integrating region-aware attention mechanisms to ensure both spatial precision and temporal coherence. To address the scarcity of high-quality Chinese training data, we introduce CHDTF, a Chinese high-definition talking face dataset. Extensive experiments show superior performance over existing methods across multiple metrics, including lip synchronization, expression quality, and temporal consistency. Project Page: https://kangweiiliu.github.io/DisentTalk.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
