基于投影的替代梯度解释用于神经编解码器包装器
A Projection-Based Surrogate Gradient Interpretation for Neural Codec Wrappers
AI总结：
针对视频编解码器不可微导致神经包装器训练困难的问题，提出将替代梯度解释为编解码器的一阶局部近似，并证明其适用于预处理和后处理网络的完整神经包装任务，在x264和VVenC上分别实现高达-23.59%和-20.07%的BD-Rate降低。
摘要：
神经包装器是学习的前处理和后处理网络，旨在增强传统视频编解码器的性能。尽管这些方法能显著提高压缩效率，但由于编码过程中涉及多个离散决策导致视频编解码器不可微，训练它们仍然具有挑战性。替代梯度最近成为一种有效的解决方案，能够实现与传统编解码器的端到端学习。它们有两个主要优势：避免训练额外的网络来模拟编解码器，并能提高压缩性能。特别是，最近提出的SCALED方法利用真实的压缩误差，在训练神经预处理器（如下采样器）方面显示出强劲效果。然而，这种SCALED梯度最初是作为重参数化技巧引入的，这限制了其可解释性。在本文中，我们表明这种替代梯度可以解释为视频编解码器的一阶局部近似，从而提供了对其有效性的洞察。我们进一步证明，它不仅适用于学习下采样操作，也适用于更困难的带有前处理和后处理网络的完整神经包装任务。最后，我们表明该方法在不同视频编解码器、质量因子和任务（包括多个下采样比例）上具有良好的泛化能力，相对于标准重采样基线，在x264上实现了高达-23.59%的BD-Rate（PSNR）降低，在VVenC上实现了高达-20.07%的降低。
英文摘要：
Neural wrappers are learned pre-and postprocessing networks designed to enhance the performance of conventional video codecs. Although these approaches can significantly improve compression efficiency, training them remains challenging due to the non-differentiability of video codecs, which arises from the multiple discrete decisions involved in the encoding process. Surrogate gradients have recently emerged as an effective solution for enabling end-to-end learning with conventional codecs. They offer two main advantages: they avoid training an additional network to mimic the codec, and they can improve compression performance. In particular, the recently proposed SCALED method, which leverages the true compression error, has shown strong results for training neural pre-processors such as downscalers. However, this SCALED gradient was originally introduced as a reparameterization trick, which limits its interpretability. In this paper, we show that this surrogate gradient can be interpreted as a first-order local approximation of the video codec, providing insight into its effectiveness. We further demonstrate that it is effective not only for learning downscaling operations, but also for the more challenging task of full neural wrapping with pre-and post-processing networks. Finally, we show that the approach generalizes well across different video codecs, quality factors, and tasks, including multiple downscaling ratios, yielding BD-Rate (PSNR) reductions of up to -23.59% on x264 and -20.07% on VVenC relative to standard resampling baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
