arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2207.08674cs.CV
收藏

基于块的时域冗余优化提高视频超分辨率

Boosting Video Super Resolution with Patch-Based Temporal Redundancy Optimization

Xi’an Jiaotong University(西安交通大学);ByteDance Intelligent Creation Lab(字节跳动智能创作实验室);Nanjing University of Science and Technology(南京理工大学)

Yuhao Huang, Hang Dong, Jinshan Pan, Chao Zhu, Yu Guo, Ding Liu, Lean Fu, Fei Wang

摘要:

现有的视频超分辨率(VSR)算法的成功主要源于利用相邻帧的时间信息。然而,这些方法都没有讨论的时间冗余的影响,在补丁与静止的目标和背景,通常使用所有的信息,在相邻的帧没有任何歧视。在本文中,我们观察到的时间冗余会带来不利影响的信息传播,这限制了大多数现有的VSR方法的性能。出于这一观察,我们的目标是改善现有的VSR算法处理的时间冗余补丁的优化方式。我们开发了两个简单而有效的即插即用的方法,以提高现有的本地和非本地传播为基础的VSR算法广泛使用的公共视频的性能。为了更全面地评估现有VSR算法的鲁棒性和性能,我们还收集了一个新的包含各种公共视频的数据集作为测试集。广泛的评估表明,所提出的方法可以显着提高现有的VSR方法从野生场景收集的视频的性能,同时保持其现有的常用数据集上的性能。该代码可在https://github.com/HYHsimon/Boosted-VSR上获得。

英文摘要:

The success of existing video super-resolution (VSR) algorithms stems mainly exploiting the temporal information from the neighboring frames. However, none of these methods have discussed the influence of the temporal redundancy in the patches with stationary objects and background and usually use all the information in the adjacent frames without any discrimination. In this paper, we observe that the temporal redundancy will bring adverse effect to the information propagation,which limits the performance of the most existing VSR methods. Motivated by this observation, we aim to improve existing VSR algorithms by handling the temporal redundancy patches in an optimized manner. We develop two simple yet effective plug and play methods to improve the performance of existing local and non-local propagation-based VSR algorithms on widely-used public videos. For more comprehensive evaluating the robustness and performance of existing VSR algorithms, we also collect a new dataset which contains a variety of public videos as testing set. Extensive evaluations show that the proposed methods can significantly improve the performance of existing VSR methods on the collected videos from wild scenarios while maintain their performance on existing commonly used datasets. The code is available at https://github.com/HYHsimon/Boosted-VSR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑