arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2603.14331cs.CV

AvatarForcing:通过局部-未来滑动窗口去噪实现一步流式谈话 avatars

AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising

Zhejiang University(浙江大学);Beijing University of Posts and Telecommunications(北京邮电大学);Kling Team, Kuaishou Technology(快手科技 Kling 团队);Tsinghua University(清华大学)

Liyuan Cui, Wentao Hu, Wenyuan Zhang, Zesong Yang, Fan Shi, Xiaoqiang Liu

AI总结:

本文提出AvatarForcing,一种一步流式扩散框架,通过局部-未来滑动窗口去噪实现高实时性谈话 avatars,实验证明其在34ms/帧下具备高质量视觉和唇同步性能。

摘要:

实时谈话 avatars 生成需要低延迟和分钟级时间稳定性。自回归(AR)强制可实现流式推理,但存在曝光偏差,导致错误累积且不可逆。相比之下,全序列扩散变换器缓解漂移但计算成本过高。我们提出AvatarForcing,一种一步流式扩散框架,通过异构噪声水平对固定局部-未来窗口进行去噪,并在每一步以恒定成本生成一个干净块。为稳定无界流,方法引入双锚点时间强制:风格锚点重索引RoPE以保持与活跃窗口的固定相对位置,并应用锚点音频零填充;时间锚点重用最近生成的干净块以确保平滑过渡。实时一步推理通过两阶段流式蒸馏与离线ODE回填和分布匹配实现。在标准基准和新400视频长形式基准上的实验表明,使用13亿参数学生模型,在34ms/帧下实现了高质量视觉和唇同步性能。我们的页面可在:https://cuiliyuan121.github.io/AvatarForcing/

英文摘要:

Real-time talking avatar generation requires low latency and minute-level temporal stability. Autoregressive (AR) forcing enables streaming inference but suffers from exposure bias, which causes errors to accumulate and become irreversible over long rollouts. In contrast, full-sequence diffusion transformers mitigate drift but remain computationally prohibitive for real-time long-form synthesis. We present AvatarForcing, a one-step streaming diffusion framework that denoises a fixed local-future window with heterogeneous noise levels and emits one clean block per step under constant per-step cost. To stabilize unbounded streams, the method introduces dual-anchor temporal forcing: a style anchor that re-indexes RoPE to maintain a fixed relative position with respect to the active window and applies anchor-audio zero-padding, and a temporal anchor that reuses recently emitted clean blocks to ensure smooth transitions. Real-time one-step inference is enabled by two-stage streaming distillation with offline ODE backfill and distribution matching. Experiments on standard benchmarks and a new 400-video long-form benchmark show strong visual quality and lip synchronization at 34 ms/frame using a 1.3B-parameter student model for realtime streaming. Our page is available at: https://cuiliyuan121.github.io/AvatarForcing/

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑