AI 中文总结
LeapTalk是一种新型说话头生成框架，通过单步前向传播结合数据到数据传输、异质蒸馏与音频驱动无分类器引导，实现了200 FPS的稳定实时生成，打破了延迟-质量权衡。
AI 中文摘要
长时长实时说话头生成因存在延迟-质量权衡而颇具挑战：低效的多步扩散模型无法实现流式生成，而实时自回归方法则会遭遇误差累积和身份漂移问题。为解决这一缺陷，我们提出了LeapTalk，这是一种新颖的框架，仅需一次前向传播即可实现稳定且实时的说话头生成，可扩展至任意长的视频。我们方法的核心在于单步桥接蒸馏方案。一方面，我们摒弃了传统的噪声到数据范式，引入了基于布朗桥的数据到数据传输公式，以持久参考为锚点，该策略可有效缓解身份漂移并提升长期时间稳定性。另一方面，为实现从预训练扩散教师模型到学生桥接模型的平滑知识迁移，我们探索了异质蒸馏框架，该框架采用SNR对齐的时间变换Φ(τ)，以弥合两个模型之间的功能差异。此外，我们还提出了音频驱动的无分类器引导机制，以在步数大幅减少的情况下保持精细的唇同步。大量实验表明，我们的方法仅需1步即可实现高保真且时间一致的视频生成，帧率最高可达200 FPS，在效率和稳定性方面显著优于现有方法。项目页面：this https URL
英文摘要
Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation $Φ(τ)$, which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/
