LightVAE：迈向紧凑高效的视频自编码器
LightVAE: Towards Compact and Efficient Video Autoencoders
AI总结：
LightVAE是一种利用时间建模与网络深度冗余的视频VAE压缩框架，通过重参数化、时间低秩压缩等实现解码加速，在多平台多数据集上性能优于现有最优方法。
摘要：
潜在扩散模型可实现高质量视频生成，但视频VAE的编码与解码仍存在计算开销过高的问题。现有方法采用特定阶段的轻量算子，但要么同时约束空间与时间计算，要么完全移除时间计算，未区分预训练Conv3D中二者的压缩敏感性。为解决这些局限，本文提出LightVAE，一种利用时间建模与网络深度冗余的压缩框架。LightVAE将符合条件的因果Conv3D层精确重参数化为聚合空间响应与差分驱动的时间修正项。基于二者不同的压缩敏感性，时间低秩压缩（TLR）保持聚合空间核为稠密形式，仅对时间修正项应用特定层的低秩近似；代数等价的折叠实现将算术节省转化为实际加速。LightVAE还采用保持宽度的块剪枝减少网络深度，并在冻结教师后缀的反馈引导下，通过探针引导的特征对齐恢复压缩后的质量，该框架也可扩展至视频编码器。在三种视频VAE骨干网络与两种GPU平台上，LightVAE均实现了解码加速：对于Wan VAE系列，在NVIDIA H100与RTX 5090上，其解码吞吐量分别达到现有最优高效VAE解码器的2.24倍与1.11倍，同时具备更高的DAVIS PSNR与更低的LPIPS；在MiniMax-H3上，其解码吞吐量最高达到原模型的2.97倍，同时保留了原模型99.51%的VBench-2.0总分。本文代码可在指定URL获取。
英文摘要：
Latent diffusion models enable high-quality video generation, but video VAE encoding and decoding remain computationally expensive. Existing methods use stage-specific lightweight operators, yet either constrain spatial and temporal computation jointly or remove temporal computation altogether, without distinguishing their compression sensitivities in pretrained Conv3D. To address these limitations, we present LightVAE, a compression framework that exploits redundancy in temporal modeling and network depth. LightVAE exactly reparameterizes eligible causal Conv3D layers into an aggregate spatial response and a difference-driven temporal correction. Based on their distinct compression sensitivities, temporal low-rank compression (TLR) keeps the aggregate spatial kernel dense and applies layer-specific low-rank approximation only to the temporal correction. An algebraically equivalent folded implementation converts the arithmetic savings into practical acceleration. LightVAE further uses width-preserving block pruning to reduce depth and probe-guided feature alignment, guided by feedback from a frozen teacher suffix, to recover quality after compression. The framework also extends to video encoders. Across three video VAE backbones and two GPU platforms, LightVAE consistently accelerates decoding. For the Wan VAE family, it achieves up to $2.24\times$ and $1.11\times$ the decoding throughput of state-of-the-art efficient VAE decoders on NVIDIA H100 and RTX~5090, respectively, with higher DAVIS PSNR and lower LPIPS. On MiniMax-H3, it achieves up to $2.97\times$ the original decoding throughput while retaining 99.51\% of the original model's overall VBench-2.0 score. Our code is available at this https URL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
