生成式预训练自回归扩散Transformer
Generative Pre-trained Autoregressive Diffusion Transformer
AI总结：
本文提出GPDiT，一种在连续潜在空间内结合扩散与自回归的生成式预训练Transformer，通过扩散损失预测未来帧，实现高质量长程视频合成，并提升表示与少样本学习能力。
摘要：
在这项工作中，我们提出了GPDiT，一种生成式预训练自回归扩散Transformer，它在连续潜在空间内统一了扩散模型和自回归建模的优势，用于长程视频合成。GPDiT不是预测离散令牌，而是使用扩散损失自回归地预测未来的潜在帧，从而能够自然地建模运动动态和跨帧的语义一致性。这种连续自回归框架不仅提高了生成质量，还赋予了模型表示能力。此外，我们引入了一种轻量级因果注意力变体和一种基于旋转的无参数时间条件机制，提高了训练和推理效率。大量实验表明，GPDiT在视频生成质量、视频表示能力和少样本学习任务上均取得了强劲性能，突显了其作为连续空间中视频建模有效框架的潜力。
英文摘要：
In this work, we present GPDiT, a Generative Pre-trained Autoregressive Diffusion Transformer that unifies the strengths of diffusion and autoregressive modeling for long-range video synthesis, within a continuous latent space. Instead of predicting discrete tokens, GPDiT autoregressively predicts future latent frames using a diffusion loss, enabling natural modeling of motion dynamics and semantic consistency across frames. This continuous autoregressive framework not only enhances generation quality but also endows the model with representation capabilities. Additionally, we introduce a lightweight causal attention variant and a parameter-free rotation-based time-conditioning mechanism, improving both the training and inference efficiency. Extensive experiments demonstrate that GPDiT achieves strong performance in video generation quality, video representation ability, and few-shot learning tasks, highlighting its potential as an effective framework for video modeling in continuous space.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
