VideoAR: 通过下一帧与尺度预测实现自回归视频生成
VideoAR: Autoregressive Video Generation via Next-Frame & Scale Prediction
AI总结：
VideoAR通过多尺度下一帧预测与自回归建模，实现高效、一致的视频生成，提升性能并减少计算成本。
摘要：
近期视频生成的进展主要由扩散模型和流匹配模型主导，这些模型能够生成高质量的结果，但计算成本高且难以扩展。在本工作中，我们引入了VideoAR，这是首个大规模视觉自回归（VAR）框架，用于视频生成，结合了多尺度下一帧预测与自回归建模。VideoAR通过整合帧内VAR建模与因果下一帧预测，解耦空间和时间依赖性，支持一个3D多尺度分词器，高效编码时空动态。为了提高长期一致性，我们提出了多尺度时间RoPE、跨帧误差校正和随机帧掩码，这些方法共同缓解了误差传播并稳定了时间一致性。我们的多阶段预训练流程逐步在增加的分辨率和持续时间下对齐空间和时间学习。实证结果表明，VideoAR在自回归模型中取得了新的最先进的结果，其在UCF-101上的FVD从99.5提升到88.6，同时将推理步骤减少了超过10倍，并达到VBench分数81.74，与基于扩散模型的性能相差一个数量级。这些结果表明，VideoAR缩小了自回归和扩散范式的性能差距，为未来的视频生成研究提供了可扩展、高效且时间一致的基础。
英文摘要：
Recent advances in video generation have been dominated by diffusion and flow-matching models, which produce high-quality results but remain computationally intensive and difficult to scale. In this work, we introduce VideoAR, the first large-scale Visual Autoregressive (VAR) framework for video generation that combines multi-scale next-frame prediction with autoregressive modeling. VideoAR disentangles spatial and temporal dependencies by integrating intra-frame VAR modeling with causal next-frame prediction, supported by a 3D multi-scale tokenizer that efficiently encodes spatio-temporal dynamics. To improve long-term consistency, we propose Multi-scale Temporal RoPE, Cross-Frame Error Correction, and Random Frame Mask, which collectively mitigate error propagation and stabilize temporal coherence. Our multi-stage pretraining pipeline progressively aligns spatial and temporal learning across increasing resolutions and durations. Empirically, VideoAR achieves new state-of-the-art results among autoregressive models, improving FVD on UCF-101 from 99.5 to 88.6 while reducing inference steps by over 10x, and reaching a VBench score of 81.74-competitive with diffusion-based models an order of magnitude larger. These results demonstrate that VideoAR narrows the performance gap between autoregressive and diffusion paradigms, offering a scalable, efficient, and temporally consistent foundation for future video generation research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
