UniTemp: 通过双向蒸馏实现任意时间顺序的视频生成
UniTemp: Unlocking Video Generation in Any Temporal Order via Bidirectional Distillation
AI总结：
提出UniTemp框架，通过双向蒸馏训练单个自回归模型，支持任意时间方向（前向、后向、中间插值）的视频生成，解决因果3D VAE在后向生成中的不连续性，提升可控性。
摘要：
自回归视频扩散模型已成为长视频生成的一种有前景的方法，在流式设置中表现出色。然而，现有方法仅限于前向时间生成，而实际视频创作通常需要灵活的生成顺序，例如，基于未来上下文进行后向扩展，或基于过去和未来上下文进行中间插值生成。我们通过训练一个支持任意时间方向生成的自回归模型来弥合这一差距。一个关键的技术挑战来自视频扩散模型中广泛使用的因果3D VAE，它编码的潜变量严格依赖于过去上下文。虽然这种因果结构适合前向生成，但在后向生成时会导致块间不连续性。为了解决这个问题，我们引入了块级锚点潜变量，这是一组辅助潜变量，用于在后向生成过程中恢复块边界处缺失的过去上下文。基于这一设计，我们提出了UniTemp，一个双向蒸馏框架，训练单个自回归学生模型用于任意方向的视频生成。在推理时，UniTemp可以基于任意过去和/或未来帧进行条件生成，提高了双向和中间插值生成的可控性。实验表明，与仅前向方法相比，UniTemp在短和长视频生成上保持了竞争性能，同时支持多种工作流程，如双向视频扩展、中间插值生成、循环视频生成、场景转换和视觉故事生成。项目网站：此 https URL
英文摘要：
Autoregressive video diffusion models have emerged as a promising approach for long video generation, achieving strong performance in streaming settings. However, existing methods are restricted to forward temporal generation, whereas practical video creation often requires flexible generation order, e.g., conditioning on future context to extend backward, or on both past and future context for inbetween generation. We bridge this gap by training an autoregressive model that supports generation in arbitrary temporal directions. A key technical challenge arises from the Causal 3D VAE widely used in video diffusion models, which encodes latents strictly conditioned on past context. While suited for forward generation, this causal structure causes inter-block discontinuities when generation proceeds backward. To address this, we introduce blockwise anchor latents, a set of auxiliary latents that restore the missing past context at block boundaries during backward generation. Built on this design, we propose UniTemp, a bidirectional distillation framework that trains a single autoregressive student model for any-direction video generation. At inference time, UniTemp conditions on arbitrary past and/or future frames, improving controllability for both bidirectional and inbetween generation. Experiments show that UniTemp maintains competitive performance on short and long video generation compared to forward-only methods, while enabling diverse workflows such as bidirectional video extension, inbetween generation, looping video generation, scene transition, and visual story generation. Project website: https://lzhangbj.github.io/projects/unitemp/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
