基于仿射变换的深度帧预测
Affine Transformation-Based Deep Frame Prediction
摘要:
我们提出了一个神经网络模型来估计当前帧从两个参考帧,使用仿射变换和自适应空间变化滤波器。与现有的深度帧预测方法相比,估计的仿射变换允许使用更短的滤波器。预测帧被用作用于编码当前帧的参考。由于所提出的模型在编码器和解码器处都可用,因此不需要对预测帧的运动信息进行编码或传输。通过使用扩张卷积和减少滤波器长度,我们的模型比以前关于这个主题的任何神经网络都要小得多,但更准确。所提出的模型的两个版本-一个用于单向,一个用于双向预测-使用基于离散余弦变换(DCT)的l1损失与各种变换大小,多尺度均方误差(MSE)损失,和对象上下文重建损失的组合进行训练。训练的模型与HEVC视频编码流水线集成。实验表明,所提出的模型实现约7.3%,5.4%,和4.2%的平均位节省的亮度分量在低延迟P,低延迟,和随机存取配置,分别。
英文摘要:
We propose a neural network model to estimate the current frame from two reference frames, using affine transformation and adaptive spatially-varying filters. The estimated affine transformation allows for using shorter filters compared to existing approaches for deep frame prediction. The predicted frame is used as a reference for coding the current frame. Since the proposed model is available at both encoder and decoder, there is no need to code or transmit motion information for the predicted frame. By making use of dilated convolutions and reduced filter length, our model is significantly smaller, yet more accurate, than any of the neural networks in prior works on this topic. Two versions of the proposed model - one for uni-directional, and one for bi-directional prediction - are trained using a combination of Discrete Cosine Transform (DCT)-based l1-loss with various transform sizes, multi-scale Mean Squared Error (MSE) loss, and an object context reconstruction loss. The trained models are integrated with the HEVC video coding pipeline. The experiments show that the proposed models achieve about 7.3%, 5.4%, and 4.2% bit savings for the luminance component on average in the Low delay P, Low delay, and Random access configurations, respectively.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。