GLOBER：通过全局引导视频解码器实现连贯的非自回归视频生成
GLOBER: Coherent Non-autoregressive Video Generation via GLOBal Guided Video DecodER
AI总结：
提出非自回归视频生成方法 GLOBER，通过全局特征引导、扩散视频解码器、归一化帧索引与对抗损失，兼顾视频全局连贯性和局部真实性，并在多个基准上达到最先进水平。
摘要：
视频生成既需要全局连贯性，也需要局部真实性。本文提出了一种新颖的非自回归方法 GLOBER，该方法首先生成全局特征以获得全面的全局引导，然后基于全局特征合成视频帧，从而生成连贯的视频。具体而言，我们提出了一种视频自动编码器：其中视频编码器将视频编码为全局特征，而基于扩散模型构建的视频解码器则解码全局特征，并以非自回归方式合成视频帧。为实现最大灵活性，我们的视频解码器通过归一化帧索引感知时间信息，使其能够合成具有预定起始和结束帧索引的任意子视频片段。此外，引入了一种新颖的对抗损失，以提升合成视频帧之间的全局连贯性和局部真实性。最后，我们采用基于扩散的视频生成器拟合视频编码器输出的全局特征，用于视频生成。大量实验结果证明了本文所提方法的有效性和效率，并在多个基准测试上取得了新的最先进结果。
英文摘要：
Video generation necessitates both global coherence and local realism. This work presents a novel non-autoregressive method GLOBER, which first generates global features to obtain comprehensive global guidance and then synthesizes video frames based on the global features to generate coherent videos. Specifically, we propose a video auto-encoder, where a video encoder encodes videos into global features, and a video decoder, built on a diffusion model, decodes the global features and synthesizes video frames in a non-autoregressive manner. To achieve maximum flexibility, our video decoder perceives temporal information through normalized frame indexes, which enables it to synthesize arbitrary sub video clips with predetermined starting and ending frame indexes. Moreover, a novel adversarial loss is introduced to improve the global coherence and local realism between the synthesized video frames. Finally, we employ a diffusion-based video generator to fit the global features outputted by the video encoder for video generation. Extensive experimental results demonstrate the effectiveness and efficiency of our proposed method, and new state-of-the-art results have been achieved on multiple benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
