RepForcing：用于自回归视频蒸馏的表示空间初始化
RepForcing: Representation-Space Initialization for Autoregressive Video Distillation
AI总结：
RepForcing是一种用于自回归视频蒸馏的表示空间初始化方法，通过表示空间监督替代中间教师轨迹回归，在4步采样下，经分布匹配蒸馏后，可提升视频生成的VBench总分与VisionReward指标。
摘要：
将预训练视频扩散模型蒸馏为少步自回归（AR）生成器时，通常会结合学生模型初始化与分布匹配蒸馏（DMD）。中间自回归教师可为常微分方程（ODE）初始化提供因果兼容的目标，但仅靠因果兼容目标无法确保短采样步数下的生成质量。我们分析了直接潜在回归所偏好的后验均值预测，经实验发现，因果ODE初始化虽能减轻模糊问题，但仍无法解决精细细节缺失的问题。为应对这一局限，我们提出了Representation Forcing（RepForcing），该方法通过表示空间监督而非中间教师轨迹回归来学习少步因果初始化。学生模型采用教师强制训练，通过冻结视频扩散Transformer的归一化特征比较干净预测与训练目标。此阶段复用了用于自回归教师训练的双向模型生成的合成视频，替代了教师训练与因果ODE蒸馏，无需额外生成因果ODE轨迹，后续的非对称DMD流程保持不变。实验结果表明，在4个采样步数下，RepForcing在DMD前比基于中间教师的初始化更好地保留了生成质量；经DMD后，我们的4步分块模型在VBench总分上从84.04提升至84.66（+0.62分），VisionReward从6.326提升至10.074（+59.25%），优于已报道的基于因果ODE的基线模型。
英文摘要：
Distilling pretrained video diffusion models into few-step autoregressive (AR) generators typically combines student initialization with distribution matching distillation (DMD). An intermediate AR teacher can provide causally compatible targets for ODE initialization. However, causally compatible targets alone do not ensure high-quality generation under a short sampling schedule. We analyze the posterior-mean predictions favored by direct latent regression and empirically find that causal ODE initialization alleviates blur but still leaves fine detail unresolved. To address this limitation, we propose Representation Forcing, which learns few-step causal initialization through representation-space supervision rather than intermediate-teacher trajectory regression. The student is trained with teacher forcing by comparing clean predictions and training targets through normalized features of a frozen video diffusion transformer. This phase reuses the synthetic videos generated by a bidirectional model for AR teacher training and replaces both teacher training and causal ODE distillation, without generating additional causal ODE trajectories. The subsequent asymmetric DMD procedure remains unchanged. Empirical results show that, at four sampling steps, RepForcing better preserves generation quality than intermediate-teacher-based initialization before DMD. After DMD, our four-step chunk-wise model improves VBench Total from 84.04 to 84.66 (+0.62 points) and VisionReward from 6.326 to 10.074 (+59.25%) over the reported causal-ODE-based baseline.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
