STARFlow-V：基于归一化流的端到端视频生成模型
STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows
AI总结：
STARFlow-V基于归一化流提出端到端视频生成模型，具备端到端学习、鲁棒因果预测和原生似然估计等优势，实现了高质量自回归视频生成。
摘要：
归一化流（NFs）是用于连续数据的端到端似然基于生成模型，近年来在图像生成领域取得了令人鼓舞的进展。然而在视频生成领域，由于时空复杂性和计算成本显著更高，最先进的系统几乎 exclusively 依赖于扩散模型。在本文中，我们重新审视这一设计空间，提出了STARFlow-V，一种基于归一化流的视频生成器，具有诸如端到端学习、鲁棒因果预测和原生似然估计等显著优势。基于最近提出的STARFlow，STARFlow-V在时空潜在空间中采用全局-局部架构，在保持帧内丰富局部交互的同时，将因果依赖限制在全局潜在空间中。这减轻了标准自回归扩散模型生成中常见的误差积累问题。此外，我们提出了流-分数匹配，该方法为模型配备了轻量级的因果去噪器，以在自回归方式下提高视频生成的一致性。为了提高采样效率，STARFlow-V采用了一种视频感知的Jacobi迭代方案，将内部更新重述为可并行化的迭代，而不破坏因果性。得益于可逆结构，同一模型可以原生支持文本到视频、图像到视频以及视频到视频的生成任务。实验表明，STARFlow-V在视觉保真度和时间一致性方面表现强劲，与基于扩散的基线相比具有实际的采样吞吐量。这些结果为我们的知识提供了第一次证据，表明NFs能够实现高质量的自回归视频生成，从而确立了它们作为构建世界模型的研究方向。代码和生成样本可在https://github.com/apple/ml-starflow获取。
英文摘要：
Normalizing flows (NFs) are end-to-end likelihood-based generative models for continuous data, and have recently regained attention with encouraging progress on image generation. Yet in the video generation domain, where spatiotemporal complexity and computational cost are substantially higher, state-of-the-art systems almost exclusively rely on diffusion-based models. In this work, we revisit this design space by presenting STARFlow-V, a normalizing flow-based video generator with substantial benefits such as end-to-end learning, robust causal prediction, and native likelihood estimation. Building upon the recently proposed STARFlow, STARFlow-V operates in the spatiotemporal latent space with a global-local architecture which restricts causal dependencies to a global latent space while preserving rich local within-frame interactions. This eases error accumulation over time, a common pitfall of standard autoregressive diffusion model generation. Additionally, we propose flow-score matching, which equips the model with a light-weight causal denoiser to improve the video generation consistency in an autoregressive fashion. To improve the sampling efficiency, STARFlow-V employs a video-aware Jacobi iteration scheme that recasts inner updates as parallelizable iterations without breaking causality. Thanks to the invertible structure, the same model can natively support text-to-video, image-to-video as well as video-to-video generation tasks. Empirically, STARFlow-V achieves strong visual fidelity and temporal consistency with practical sampling throughput relative to diffusion-based baselines. These results present the first evidence, to our knowledge, that NFs are capable of high-quality autoregressive video generation, establishing them as a promising research direction for building world models. Code and generated samples are available at https://github.com/apple/ml-starflow.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
