AsymTalker：通过不对称蒸馏实现身份一致的长期说话头生成
AsymTalker: Identity-Consistent Long-Term Talking Head Generation via Asymmetric Distillation
AI总结：
本文提出AsymTalker，通过时空对齐和不对称蒸馏解决长期说话头生成中的身份漂移问题，实现高保真身份一致的600秒视频生成，达到66 FPS的实时推理速度。
摘要：
基于扩散的说话头生成在视觉质量上取得了显著成就，但将其扩展到长视频仍具挑战性。广泛采用的分块方法引入了两个根本性问题：（1）静态身份参考与动态音频流之间的时空错位；（2）通过自我生成的连续参考在分块间传播的身份漂移。为了解决这两个问题，我们提出AsymTalker，一种新的基于扩散的说话头生成方法，包含时间参考编码（TRE）和不对称知识蒸馏（AKD）。首先，TRE通过将静态身份图像转换为时间一致的潜在表示，通过编码时间重复的伪视频来缓解时空错位，而无需引入额外参数。其次，AKD解决分块训练中的固有条件困境：使用真实参考导致训练-推理不匹配，而自我生成的参考使监督与身份漂移纠缠。我们的不对称设计通过将教师模型锚定在真实连续参考上，提供无漂移的分块级监督，从而避免教师瓶颈。同时，学生模型在推理对齐条件下学习，仅基于自我生成的参考，并通过分布匹配训练以在长时间范围内保持身份一致性。大量实验表明，AsymTalker在HDTF和VFHQ上实现了最先进的结果。它保证了600秒视频的高保真、身份一致的合成，并达到了66 FPS的实时推理速度。
英文摘要：
Diffusion-based talking head generation has achieved remarkable visual quality, yet scaling it to long-term videos remains challenging. The widely adopted chunk-wise paradigm introduces two fundamental failures: (1) temporal-spatial misalignment between static identity references and dynamic audio streams, and (2) cascading identity drift propagated through self-generated continuity references across chunks. To address both issues, we propose AsymTalker, a novel diffusion-based talking head generation method comprising Temporal Reference Encoding (TRE) and Asymmetric Knowledge Distillation (AKD). First, TRE mitigates temporal-spatial misalignment by transforming the static identity image into a temporally coherent latent representation through encoding of a temporally replicated pseudo-video, without introducing additional parameters. Second, AKD resolves the inherent conditioning dilemma in chunk-wise training: using ground-truth references causes train-inference mismatch, while self-generated references entangle supervision with identity drift. Our asymmetric design circumvents this by anchoring the teacher model with ground-truth continuity references to provide drift-free, chunk-level supervision, thereby avoiding the teacher bottleneck. Meanwhile, the student model learns under inference-aligned conditions, conditioned only on self-generated references, and is trained via distribution matching to preserve identity over long horizons. Extensive experiments show AsymTalker achieves state-of-the-art results on HDTF and VFHQ. It guarantees high-fidelity, identity-consistent synthesis over 600-second videos and reaches a real-time inference speed of 66 FPS.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
