历史引导的视频扩散
History-Guided Video Diffusion
AI总结：
针对可变长度历史条件下视频扩散难以应用 CFG 的问题，本文提出 DFoT 架构与训练目标，并引入 History Guidance，以提升长视频质量、时序一致性、运动动态及分布外泛化能力。
摘要：
无分类器引导（CFG）是改进扩散模型中条件生成的一项关键技术，能够在提升样本质量的同时实现更精确的控制。将该技术扩展到视频扩散是很自然的做法；视频扩散根据数量可变的上下文帧生成视频，这些上下文帧统称为历史。然而，我们发现使用可变长度历史进行引导存在两个关键挑战：一是架构仅支持固定大小的条件输入，二是实验观察到 CFG 风格的历史丢弃效果较差。为解决这些问题，我们提出 Diffusion Forcing Transformer（DFoT），这是一种视频扩散架构和具有理论基础的训练目标，二者共同支持基于灵活数量历史帧的条件生成。随后，我们介绍 History Guidance，这是一类由 DFoT 独特支持的引导方法。我们表明，其最简单形式 vanilla history guidance 已能显著提升视频生成质量和时间一致性。更先进的方法 history guidance across time and frequency 进一步增强了运动动态，支持对分布外历史的组合泛化，并能够稳定生成极长视频。项目网站：https://boyuan.space/history-guidance
英文摘要：
Classifier-free guidance (CFG) is a key technique for improving conditional generation in diffusion models, enabling more accurate control while enhancing sample quality. It is natural to extend this technique to video diffusion, which generates video conditioned on a variable number of context frames, collectively referred to as history. However, we find two key challenges to guiding with variable-length history: architectures that only support fixed-size conditioning, and the empirical observation that CFG-style history dropout performs poorly. To address this, we propose the Diffusion Forcing Transformer (DFoT), a video diffusion architecture and theoretically grounded training objective that jointly enable conditioning on a flexible number of history frames. We then introduce History Guidance, a family of guidance methods uniquely enabled by DFoT. We show that its simplest form, vanilla history guidance, already significantly improves video generation quality and temporal consistency. A more advanced method, history guidance across time and frequency further enhances motion dynamics, enables compositional generalization to out-of-distribution history, and can stably roll out extremely long videos. Project website: https://boyuan.space/history-guidance
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
