AniTalker：通过身份解耦的面部运动编码生成生动多样的说话人脸
AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding
AI总结：
AniTalker提出身份解耦的面部运动编码，结合自监督学习和扩散模型，从单张人像生成多样且可控的逼真说话人脸。
摘要：
本文介绍了AniTalker，一个旨在从单张人像生成逼真说话人脸的新颖框架。与现有模型主要关注唇部同步等言语线索、无法捕捉面部表情和非言语线索的复杂动态不同，AniTalker采用了一种通用运动表示。这种创新表示能够有效捕捉广泛的面部动态，包括细微表情和头部运动。AniTalker通过两种自监督学习策略增强运动刻画：第一种策略是在同一身份内从源帧重建目标视频帧，以学习细微的运动表示；第二种策略利用度量学习开发身份编码器，同时主动最小化身份编码器与运动编码器之间的互信息。该方法确保运动表示是动态的且不含身份特定细节，显著减少了对标注数据的需求。此外，扩散模型与方差适配器的集成使得生成多样且可控的面部动画成为可能。该方法不仅展示了AniTalker创建精细逼真面部运动的能力，也凸显了其为实际应用构建动态虚拟形象的潜力。合成结果可在https://github.com/X-LANCE/AniTalker查看。
英文摘要：
The paper introduces AniTalker, an innovative framework designed to generate lifelike talking faces from a single portrait. Unlike existing models that primarily focus on verbal cues such as lip synchronization and fail to capture the complex dynamics of facial expressions and nonverbal cues, AniTalker employs a universal motion representation. This innovative representation effectively captures a wide range of facial dynamics, including subtle expressions and head movements. AniTalker enhances motion depiction through two self-supervised learning strategies: the first involves reconstructing target video frames from source frames within the same identity to learn subtle motion representations, and the second develops an identity encoder using metric learning while actively minimizing mutual information between the identity and motion encoders. This approach ensures that the motion representation is dynamic and devoid of identity-specific details, significantly reducing the need for labeled data. Additionally, the integration of a diffusion model with a variance adapter allows for the generation of diverse and controllable facial animations. This method not only demonstrates AniTalker's capability to create detailed and realistic facial movements but also underscores its potential in crafting dynamic avatars for real-world applications. Synthetic results can be viewed at https://github.com/X-LANCE/AniTalker.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
