MTC-VAE：具有内容意识的多级时间压缩
MTC-VAE: Multi-Level Temporal Compression with Content Awareness
AI总结：
MTC-VAE通过多级时间压缩技术提升视频压缩效率，结合内容意识方法实现高效压缩与性能优化。
摘要：
潜在视频扩散模型（LVDMs）依赖变分自编码器（VAEs）将视频压缩成紧凑的潜在表示。对于连续变分自编码器（VAEs），实现更高的压缩率是理想的；然而，当在不扩展隐藏通道维度的情况下添加额外的采样层时，效率会显著下降。在本文中，我们提出了一种技术，将固定压缩率的VAEs转换为支持多级时间压缩的模型，提供了一种简单且最小的微调方法，以抵消在高压缩率下的性能下降。此外，我们探讨了不同压缩级别对具有不同特征的视频片段上模型性能的影响，提供了我们所提出方法有效性的实证证据。我们还研究了我们的多级时间压缩VAE与基于扩散的生成模型DiT的整合，突显了这些框架内成功的同时训练和兼容性。这项研究展示了多级时间压缩的潜在应用。
英文摘要：
Latent Video Diffusion Models (LVDMs) rely on Variational Autoencoders (VAEs) to compress videos into compact latent representations. For continuous Variational Autoencoders (VAEs), achieving higher compression rates is desirable; yet, the efficiency notably declines when extra sampling layers are added without expanding the dimensions of hidden channels. In this paper, we present a technique to convert fixed compression rate VAEs into models that support multi-level temporal compression, providing a straightforward and minimal fine-tuning approach to counteract performance decline at elevated compression rates.Moreover, we examine how varying compression levels impact model performance over video segments with diverse characteristics, offering empirical evidence on the effectiveness of our proposed approach. We also investigate the integration of our multi-level temporal compression VAE with diffusion-based generative models, DiT, highlighting successful concurrent training and compatibility within these frameworks. This investigation illustrates the potential uses of multi-level temporal compression.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
