驯服教师强制用于掩码自回归视频生成
Taming Teacher Forcing for Masked Autoregressive Video Generation
AI总结：
提出MAGI混合视频生成框架，通过完整教师强制（CTF）改进掩码自回归，显著提升FVD分数，并能生成超100帧的连贯视频。
摘要：
我们提出了MAGI，一个混合视频生成框架，它结合了用于帧内生成的掩码建模和用于下一帧生成的因果建模。我们的关键创新，完整教师强制（CTF），将掩码帧基于完整观察帧而非掩码帧（即掩码教师强制，MTF）进行条件化，实现了从令牌级（补丁级）到帧级自回归生成的平滑过渡。CTF显著优于MTF，在首帧条件视频预测的FVD分数上实现了+23%的提升。为解决曝光偏差等问题，我们采用了针对性的训练策略，在自回归视频生成中设立了新的基准。实验表明，MAGI能够生成超过100帧的长且连贯的视频序列，即使仅用16帧进行训练，这突显了其在大规模、高质量视频生成方面的潜力。
英文摘要：
We introduce MAGI, a hybrid video generation framework that combines masked modeling for intra-frame generation with causal modeling for next-frame generation. Our key innovation, Complete Teacher Forcing (CTF), conditions masked frames on complete observation frames rather than masked ones (namely Masked Teacher Forcing, MTF), enabling a smooth transition from token-level (patch-level) to frame-level autoregressive generation. CTF significantly outperforms MTF, achieving a +23% improvement in FVD scores on first-frame conditioned video prediction. To address issues like exposure bias, we employ targeted training strategies, setting a new benchmark in autoregressive video generation. Experiments show that MAGI can generate long, coherent video sequences exceeding 100 frames, even when trained on as few as 16 frames, highlighting its potential for scalable, high-quality video generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
