Causal-rCM：一种用于流式视频生成和交互式世界模型中自回归扩散蒸馏的统一教师强制与自我强制开放配方
Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models
AI总结：
提出Causal-rCM框架，将扩散蒸馏扩展到自回归视频扩散，通过教师强制（前向散度）与自我强制（反向散度）互补，实现流式视频生成和交互式世界模型，在帧级和块级设置中达到最先进性能。
摘要：
具有因果扩散变换器的自回归视频扩散已成为实时流式视频生成和动作条件交互式世界模型的主要范式。在这项工作中，我们将rCM（一种先进的扩散蒸馏框架）扩展到自回归视频扩散。rCM的核心哲学在于扩散蒸馏中前向散度与反向散度之间的互补性，分别由一致性模型（CM）和分布匹配蒸馏（DMD）表示。这一哲学自然延续到自回归设置中，其中教师强制（TF）提供了一种离线的、前向散度的因果训练范式，而自我强制（SF）则对应于一种在策略的、反向散度的细化。我们的贡献是：（1）通过大量实验，我们表明教师强制CM目前是自我强制DMD作为初始化策略的最佳补充；（2）我们首次实现了基于教师强制的连续时间CM（例如sCM/MeanFlow）用于自回归视频扩散，这得益于我们定制的掩码FlashAttention-2 JVP内核，与离散时间CM（dCM）相比实现了10倍的收敛加速；（3）我们引入了Causal-rCM，一种领先的、统一的、可扩展的算法-基础设施开放配方，用于扩散蒸馏和因果训练；（4）我们在帧级和块级设置中实现了最先进的流式视频生成性能，仅使用合成数据进行训练。值得注意的是，我们蒸馏的2步因果Wan2.1-1.3B模型仅需1或2个采样步骤即可达到84.63的VBench-T2V分数。我们进一步将Causal-rCM应用于Cosmos 3，这是一种用于物理AI的先进全模态世界基础模型，具有动作条件生成能力，从而实现了交互式世界模型。
英文摘要：
Autoregressive video diffusion with causal diffusion transformers has emerged as a major paradigm for real-time streaming video generation and action-conditioned interactive world models. In this work, we extend rCM, an advanced diffusion distillation framework, to autoregressive video diffusion. The core philosophy of rCM lies in the complementarity between forward and reverse divergences, represented by consistency models (CMs) and distribution matching distillation (DMD), respectively, in diffusion distillation. This philosophy naturally carries over to the autoregressive setting, where teacher-forcing (TF) provides an offline, forward-divergence causal training paradigm, while self-forcing (SF) corresponds to an on-policy, reverse-divergence refinement. Our contributions are: (1) through extensive experiments, we show that teacher-forcing CM is currently the best complement to self-forcing DMD as an initialization strategy (2) we present the first implementation of teacher-forcing-based continuous-time CMs (e.g., sCM/MeanFlow) for autoregressive video diffusion, enabled by our custom-mask FlashAttention-2 JVP kernel, achieving 10$\times$ faster convergence compared to discrete-time CMs (dCMs) (3) we introduce Causal-rCM, a leading, unified, and scalable algorithm-infrastructure open recipe for diffusion distillation and causal training (4) we achieve state-of-the-art streaming video generation performance in both frame-wise and chunk-wise settings, using only synthetic data for training. Notably, our distilled 2-step causal Wan2.1-1.3B model achieves a VBench-T2V score of 84.63 with only 1 or 2 sampling steps. We further apply Causal-rCM to Cosmos 3, an advanced omnimodal world foundation model for physical AI with action-conditioned generation capability, enabling an interactive world model.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
