15 pages, 7 figures. Code and video results are available at https://desaixie.github.io/pa-vdm/. v2: Accepted to CVPRW 2025. Updated figures, tables, notations, and text in all sections. Added comparison with more baseline methods, FVD metric results, user study, and discussion on parallel works
Current frontier video diffusion models have demonstrated remarkable results at generating high-quality videos. However, they can only generate short video clips, normally around 10 seconds or 240 frames, due to computation limitations during training. Existing methods naively achieve autoregressive long video generation by directly placing the ending of the previous clip at the front of the attention window as conditioning, which leads to abrupt scene changes, unnatural motion, and error accumulation. In this work, we introduce a more natural formulation of autoregressive long video generation by revisiting the noise level assumption in video diffusion models. Our key idea is to 1. assign the frames with per-frame, progressively increasing noise levels rather than a single noise level and 2. denoise and shift the frames in small intervals rather than all at once. This allows for smoother attention correspondence among frames with adjacent noise levels, larger overlaps between the attention windows, and better propagation of information from the earlier to the later frames. Video diffusion models equipped with our progressive noise schedule can autoregressively generate long videos with much improved fidelity compared to the baselines and minimal quality degradation over time. We present the first results on text-conditioned 60-second (1440 frames) long video generation at a quality close to frontier models. Code and video results are available at https://desaixie.github.io/pa-vdm/.
Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models
Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generation
Nanjing University（南京大学）；Institute of Artificial Intelligence, China Telecom (TeleAI)（中国电信人工智能研究院）；Shanghai Jiao Tong University（上海交通大学）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；University of Chinese Academy of Sciences（中国科学院大学）
GLOBER: Coherent Non-autoregressive Video Generation via GLOBal Guided Video DecodER
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
CanvasMAR: Improving Masked Autoregressive Video Prediction With Canvas
Institute for Artificial Intelligence, Peking University, Beijing, China（人工智能研究院，北京大学，北京，中国）；School of Intelligence Science and Technology, Peking University, Beijing, China（智能科学与技术学院，北京大学，北京，中国）；State Key Laboratory of General Artificial Intelligence, Peking University, Beijing, China（通用人工智能国家重点实验室，北京大学，北京，中国）
Autoregressive Video Generation without Vector Quantization
Beijing University of Posts and Telecommunications（北京邮电大学）；Key Laboratory of Intelligent Information Processing, ICT, CAS（中国科学院计算技术研究所智能信息处理重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Dalian University of Technology（大连理工大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）
DreamForge: Motion-Aware Autoregressive Video Generation for Multi-View Driving Scenes
Zhejiang University（浙江大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；Technical University of Munich（慕尼黑技术大学）
DiffVC: A Non-autoregressive Framework Based on Diffusion Model for Video Captioning
School of Software, Northwestern Polytechnical University（西北工业大学软件学院）；School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机学院）；Beijing University Of Technology（北京工业大学）
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（中国教育部多媒体可信感知与高效计算重点实验室，厦门大学）；ByteDance（字节跳动）
DiT as Real-Time Rerenderer: Streaming Video Stylization with Autoregressive Diffusion Transformer
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；XMax.AI Ltd.（XMax人工智能有限公司）；Nanyang Technological University（南洋理工大学）
