具有解耦时间和空间上下文的自回归视频自编码器
Autoregressive Video Autoencoder with Decoupled Temporal and Spatial Context
AI总结：
ARVAE通过解耦时间和空间上下文，实现高效视频压缩与重建，具备轻量模型和小数据优势，适用于视频生成等下游任务。
摘要：
视频自编码器将视频压缩成紧凑的潜在表示，以实现高效的重建，对提高视频生成的质量和效率起着关键作用。然而，现有的视频自编码器往往将空间和时间信息纠缠在一起，限制了其捕捉时间一致性的能力，导致性能不佳。为此，我们提出了自回归视频自编码器（ARVAE），它以自回归的方式压缩和重建每一帧，条件是其前一帧，从而允许灵活处理任意长度的视频。ARVAE引入了时间-空间解耦的表示，结合下采样流场以实现时间一致性，以及空间相对补偿以补偿新出现的内容，实现了高压缩效率而无信息损失。具体而言，编码器将当前和前一帧压缩成时间运动和空间补充，而解码器则从潜在表示中重建原始帧，给定前一帧。采用多阶段训练策略逐步优化模型。广泛的实验表明，ARVAE在极轻量的模型和小规模训练数据下实现了卓越的重建质量。此外，对视频生成任务的评估突显了其在下游应用中的强大潜力。
英文摘要：
Video autoencoders compress videos into compact latent representations for efficient reconstruction, playing a vital role in enhancing the quality and efficiency of video generation. However, existing video autoencoders often entangle spatial and temporal information, limiting their ability to capture temporal consistency and leading to suboptimal performance. To address this, we propose Autoregressive Video Autoencoder (ARVAE), which compresses and reconstructs each frame conditioned on its predecessor in an autoregressive manner, allowing flexible processing of videos with arbitrary lengths. ARVAE introduces a temporal-spatial decoupled representation that combines downsampled flow field for temporal coherence with spatial relative compensation for newly emerged content, achieving high compression efficiency without information loss. Specifically, the encoder compresses the current and previous frames into the temporal motion and spatial supplement, while the decoder reconstructs the original frame from the latent representations given the preceding frame. A multi-stage training strategy is employed to progressively optimize the model. Extensive experiments demonstrate that ARVAE achieves superior reconstruction quality with extremely lightweight models and small-scale training data. Moreover, evaluations on video generation tasks highlight its strong potential for downstream applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
