arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2606.04410cs.CV
收藏

超快神经视频压缩

Ultra-Fast Neural Video Compression

Microsoft Research Asia(微软亚洲研究院);University of Science and Technology of China(中国科学技术大学)

Jiahao Li, Wenxuan Xie, Zhaoyang Jia, Bin Li, Zongyu Guo, Xiaoyi Zhang, Yan Lu

AI总结:

提出基于块的编码框架DCVC-UF,通过联合时空建模和并行重建实现超快编解码,显著提升率失真-复杂度权衡。

摘要:

尽管神经视频编解码器(NVC)已展现出优越的压缩比,但其过高的计算复杂度仍是实际部署的关键障碍。本文引入一种基于块的编码框架,旨在显著改善率失真-复杂度权衡。我们的方法不是逐帧处理,而是将多个帧组成的块编码为单个紧凑的潜在表示,并同时解码它们。这是通过用于联合时空建模的跨帧交互模块和用于并行重建的帧特定解码器实现的。这种范式不仅显著提高了编码吞吐量,还有助于更有效地建模长期时间相关性。为了进一步提高速度,我们提出了一种简化的熵编码机制,将比特流交互整合为单一步骤,大幅减少解码开销。基于这些创新,我们提出了DCVC-UF(超快),一种新的NVC,在性能上树立了新的SOTA。我们的实验表明,DCVC-UF可以实现超快的编码和解码速度,显著优于之前的领先编解码器。DCVC-UF是NVC发展历程中的一个显著里程碑。代码位于https://github.com/microsoft/DCVC。

英文摘要:

While neural video codecs (NVCs) have demonstrated superior compression ratio, their prohibitive computational complexity remains a critical barrier to real-world deployment. This paper introduces a chunk-based coding framework designed to significantly improve the rate-distortion-complexity trade-off. Instead of processing frames sequentially, our approach encodes a chunk of multiple frames into a single compact latent representation and decodes them simultaneously. This is enabled by cross-frame interaction modules for joint spatial-temporal modeling and frame-specific decoders for parallel reconstruction. This paradigm not only dramatically enhances coding throughput but also facilitates more effective modeling of long-term temporal correlations. To further boost speed, we propose a streamlined entropy coding mechanism that consolidates bit-stream interactions into a single step, substantially reducing decoding overhead. Building on these innovations, we present DCVC-UF (Ultra-Fast), a new NVC that sets a new SOTA in performance. Our experiments show that DCVC-UF can achieve ultra-fast encoding and decoding speeds, significantly outperforming previous leading codecs. DCVC-UF serves as a notable landmark in the journey of NVC evolution. The code is at https://github.com/microsoft/DCVC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑