arXiv 2103.094552021-03-17cs.CVeess.IV

预测辅助的视频流帧超分辨率

Prediction-assistant Frame Super-Resolution for Video Streaming

Intel（英特尔公司）

Wang Shen, Wenbo Bao, Guangtao Zhai, Charlie L Wang, Jerry W Hu, Zhiyong Gao

摘要：

视频帧的传输延迟在实时应用中是至关重要的，如在线视频游戏、现场直播等，新帧的接收截止时间必须赶上帧渲染时间。否则，系统会缓冲一段时间，用户会遇到屏幕冻结，导致用户体验不理想。一种有效的方法是在较差的带宽条件下以较低质量传输帧，例如使用可伸缩视频编码。在本文中，我们建议在两种情况下使用有损帧来增强视频质量。首先，当当前帧太晚而不能在渲染截止日期之前接收时（即，丢失），我们建议使用先前接收到的高分辨率图像来预测未来帧。第二，当当前接收的帧的质量较低时（即，有损），我们建议使用先前接收的高分辨率帧来增强低质量的当前帧。对于第一种情况，我们提出了一个小而有效的视频帧预测网络。对于第二种情况，我们将视频预测网络改进为视频增强网络，以关联当前帧和先前帧来恢复高质量图像。大量的实验结果表明，我们的方法表现良好，对国家的最先进的算法在有损视频流环境。

英文摘要：

Video frame transmission delay is critical in real-time applications such as online video gaming, live show, etc. The receiving deadline of a new frame must catch up with the frame rendering time. Otherwise, the system will buffer a while, and the user will encounter a frozen screen, resulting in unsatisfactory user experiences. An effective approach is to transmit frames in lower-quality under poor bandwidth conditions, such as using scalable video coding. In this paper, we propose to enhance video quality using lossy frames in two situations. First, when current frames are too late to receive before rendering deadline (i.e., lost), we propose to use previously received high-resolution images to predict the future frames. Second, when the quality of the currently received frames is low~(i.e., lossy), we propose to use previously received high-resolution frames to enhance the low-quality current ones. For the first case, we propose a small yet effective video frame prediction network. For the second case, we improve the video prediction network to a video enhancement network to associate current frames as well as previous frames to restore high-quality images. Extensive experimental results demonstrate that our method performs favorably against state-of-the-art algorithms in the lossy video streaming environment.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

摘要：

英文摘要：

相关论文