CanvasMAR: 通过Canvas提升遮蔽自回归视频预测
CanvasMAR: Improving Masked Autoregressive Video Prediction With Canvas
AI总结：
CanvasMAR通过引入canvas机制和运动感知采样顺序，提升视频生成的保真度和效率，实现更高质量的视频生成。
摘要：
遮蔽自回归模型（MAR）已作为一种强大的范式出现在图像和视频生成中，结合了遮蔽建模的灵活性和连续标记器的表达性。然而，当单独采样帧时，视频MAR模型常常产生高度失真输出，由于缺乏结构化的全局先验，尤其是在仅使用少量采样步骤时。为了解决这个问题，我们提出了CanvasMAR，一种新的自回归视频预测模型，通过引入一个canvas——一个模糊的、对下一帧的全局一步预测，作为在遮蔽生成中的非均匀掩码。canvas在采样早期提供全局结构，使帧合成更快且更连贯。为进一步稳定自回归采样，我们提出了一种易于-困难的课程学习方法，通过一种运动感知的采样顺序，先合成相对静止的区域，再处理高度动态的区域。我们还集成了组合分类器无关引导，共同加强canvas和时间条件，以提高生成保真度。在BAIR、UCF-101和Kinetics-600基准测试中，实验表明CanvasMAR在更少的自回归步骤下生成更高质量的视频。在具有挑战性的Kinetics-600数据集上，CanvasMAR在自回归模型中取得了显著性能，并与先进的扩散方法相抗衡。
英文摘要：
Masked autoregressive models (MAR) have emerged as a powerful paradigm for image and video generation, combining the flexibility of masked modeling with the expressiveness of continuous tokenizers. However, when sampling individual frames, video MAR models often produce highly distorted outputs due to the lack of a structured global prior, especially when using only a few sampling steps. To address this, we propose CanvasMAR, a novel autoregressive video prediction model that predicts high-fidelity frames with few sampling steps by introducing a canvas--a blurred, global one-step prediction of the next frame that serves as a non-uniform mask during masked generation. The canvas supplies global structure early in sampling, enabling faster and more coherent frame synthesis. To further stabilize autoregressive sampling, we propose an easy-to-hard curriculum via a motion-aware sampling order that synthesizes relatively stationary regions before attending to highly dynamic ones. We also integrate compositional classifier-free guidance that jointly strengthens the canvas and temporal conditioning to improve generation fidelity. Experiments on the BAIR, UCF-101, and Kinetics-600 benchmarks demonstrate that CanvasMAR produces higher-quality videos with fewer autoregressive steps. On the challenging Kinetics-600 dataset, CanvasMAR achieves remarkable performance among autoregressive models and rivals advanced diffusion-based methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
