Video-Mirai: 自回归视频扩散模型需要远见
Video-Mirai: Autoregressive Video Diffusion Models Need Foresight
AI总结：
提出Video-Mirai训练方法，通过冻结的远见编码器从完整生成序列中提取未来信息并蒸馏到因果状态，在不改变推理过程的情况下弥合表示层面的规划差距，提升长视频生成的一致性。
摘要：
因果视频生成器必须从过去预测，但它们不必仅从过去学习。在流式自回归视频扩散中，每个发射的片段成为未来片段必须保留的承诺。然而，标准训练只要求每个因果状态解释当前。这造成了我们称之为表示层面的规划差距：适合当前片段的状态可能丢弃未来一致性所需的身份、布局和运动信息。我们引入Video-Mirai，一种仅训练的方法，在不改变因果推理的情况下弥合这一差距：生成器因果地展开，一个冻结的远见编码器非因果地读取完成的展开，一个轻量级预测器将得到的停止梯度目标蒸馏到因果状态。未来帧监督表示，从不监督生成器输入。在推理时，编码器和预测器被丢弃，原始架构、每步FLOPs和KV缓存行为保持不变。Video-Mirai在5秒VBench上将强因果强制基线从83.8提高到84.6（总分）。在超出训练范围的30秒展开中，主体一致性从84.9提高到88.5，背景一致性从90.2提高到91.9。消融实验确定未来条件目标是关键因素，探针实验显示未来帧从当前特征中更易解码。因果性应约束推理，而非表示监督。我们的研究强调视觉自回归模型需要远见。项目页面：此https URL。
英文摘要：
Causal video generators must predict from the past, but they need not learn only from it. In streaming autoregressive video diffusion, each emitted segment becomes a commitment that future segments must preserve. Standard training, however, only asks each causal state to explain the present. This creates what we call a representation-level planning gap: states that fit the current segment may discard identity, layout, and motion information needed for a consistent future. We introduce Video-Mirai, a training-only method that closes this gap without changing causal inference: the generator rolls out causally, a frozen foresight encoder reads the completed rollout non-causally, and a lightweight predictor distills the resulting stopped-gradient targets into causal states. Future frames supervise representations, never generator inputs. At inference, the encoder and predictor are discarded, leaving the original architecture, per-step FLOPs, and KV-cache behavior unchanged. Video-Mirai improves a strong Causal-Forcing baseline on 5-second VBench from 83.8 to 84.6 in terms of Total Score. On 30-second rollouts beyond the training horizon, subject consistency improves from 84.9 to 88.5 and background consistency from 90.2 to 91.9. Ablations identify future-conditioned targets as the key ingredient, and probes show that future frames become more decodable from current features. Causality should constrain inference, not representation supervision. Our study highlights that visual autoregressive models need foresight. Project page: https://y0uroy.github.io/Video-Mirai.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
