arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2402.17414cs.CVeess.IV
收藏

基于特征调制的神经视频压缩

Neural Video Compression with Feature Modulation

Microsoft Research Asia(微软亚洲研究院)

Jiahao Li, Bin Li, Yan Lu

AI总结:

本文提出基于特征调制的条件编码NVC,通过可学习量化缩放器和周期性时间特征刷新,解决单模型宽质量范围与长预测链质量退化问题,并显著节省码率和计算量。

摘要:

新兴的基于条件编码的神经视频编解码器(NVC)相比常用的基于残差编码的编解码器展现出优势,而且最新的NVC已经宣称性能超过最佳传统编解码器。然而,仍存在阻碍NVC实用化的关键问题。本文提出一种强大的基于条件编码的NVC,通过特征调制解决两个关键问题。第一个问题是如何在单一模型中支持较宽的质量范围。此前具备该能力的NVC平均仅支持约3.8 dB的PSNR范围。为克服这一限制,我们通过可学习的量化缩放器调制当前帧的潜在特征。在训练期间,我们专门设计了均匀量化参数采样机制,以增强编码与量化之间的协调性。这使量化缩放器得到更好的学习,并帮助我们的NVC支持约11.4 dB的PSNR范围。第二个问题是如何让NVC在长预测链下仍能正常工作。我们揭示,此前的SOTA NVC在使用较大的帧内周期设置时存在明显的质量下降问题。为此,我们提出利用周期性刷新机制调制时间特征,以提升质量。值得注意的是,在单帧内设置下,我们的编解码器相比此前的SOTA NVC可实现29.7%的码率节省,同时MACs降低16%。我们的编解码器是NVC演进历程中的一个重要里程碑。代码位于 https://github.com/microsoft/DCVC。

英文摘要:

The emerging conditional coding-based neural video codec (NVC) shows superiority over commonly-used residual coding-based codec and the latest NVC already claims to outperform the best traditional codec. However, there still exist critical problems blocking the practicality of NVC. In this paper, we propose a powerful conditional coding-based NVC that solves two critical problems via feature modulation. The first is how to support a wide quality range in a single model. Previous NVC with this capability only supports about 3.8 dB PSNR range on average. To tackle this limitation, we modulate the latent feature of the current frame via the learnable quantization scaler. During the training, we specially design the uniform quantization parameter sampling mechanism to improve the harmonization of encoding and quantization. This results in a better learning of the quantization scaler and helps our NVC support about 11.4 dB PSNR range. The second is how to make NVC still work under a long prediction chain. We expose that the previous SOTA NVC has an obvious quality degradation problem when using a large intra-period setting. To this end, we propose modulating the temporal feature with a periodically refreshing mechanism to boost the quality. %Besides solving the above two problems, we also design a single model that can support both RGB and YUV colorspaces. Notably, under single intra-frame setting, our codec can achieve 29.7\% bitrate saving over previous SOTA NVC with 16\% MACs reduction. Our codec serves as a notable landmark in the journey of NVC evolution. The codes are at https://github.com/microsoft/DCVC.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑