arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2104.07473cs.CVcs.AIcs.LGcs.MMeess.IV
收藏

缩放SlowMo:一种有效的空时视频超分辨率一步法框架

Zooming SlowMo: An Efficient One-Stage Framework for Space-Time Video Super-Resolution

Purdue University(普渡大学);University of Rochester(罗切斯特大学);Northeastern University(东北大学)

Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P. Allebach, Chenliang Xu

摘要:

在本文中,我们提出了空时视频超分辨率,其目的是从低分辨率(LR)和低帧速率(LFR)的视频序列生成高分辨率(HR)的慢动作视频。一种简单的方法是将其分解为两个子任务:视频帧插值(VFI)和视频超分辨率(VSR)。然而,时间内插和空间上尺度在这个问题中是内部相关的。两阶段办法不能充分利用这一自然属性。此外,最先进的VFI或VSR深度网络通常具有大型帧重建模块,以获得高质量的照片级逼真视频帧,这使得两阶段方法具有大型模型,因此相对耗时。为了克服这些问题,我们提出了一个单级时空视频超分辨率框架,它可以直接从输入LR和LFR视频重建HR慢动作视频序列。而不是重建丢失的LR中间帧的VFI模型做,我们在时间上插值LR帧的丢失LR帧捕获本地时间上下文的特征时间插值模块。在广泛使用的基准测试上进行的大量实验表明,所提出的框架不仅在干净和有噪声的LR帧上实现了更好的定性和定量性能,而且比最近最先进的两阶段网络快几倍。源代码发布在https://github.com/Mukosame/Zooming-Slow-Mo-CVPR-2020上。

英文摘要:

In this paper, we address the space-time video super-resolution, which aims at generating a high-resolution (HR) slow-motion video from a low-resolution (LR) and low frame rate (LFR) video sequence. A naïve method is to decompose it into two sub-tasks: video frame interpolation (VFI) and video super-resolution (VSR). Nevertheless, temporal interpolation and spatial upscaling are intra-related in this problem. Two-stage approaches cannot fully make use of this natural property. Besides, state-of-the-art VFI or VSR deep networks usually have a large frame reconstruction module in order to obtain high-quality photo-realistic video frames, which makes the two-stage approaches have large models and thus be relatively time-consuming. To overcome the issues, we present a one-stage space-time video super-resolution framework, which can directly reconstruct an HR slow-motion video sequence from an input LR and LFR video. Instead of reconstructing missing LR intermediate frames as VFI models do, we temporally interpolate LR frame features of the missing LR frames capturing local temporal contexts by a feature temporal interpolation module. Extensive experiments on widely used benchmarks demonstrate that the proposed framework not only achieves better qualitative and quantitative performance on both clean and noisy LR frames but also is several times faster than recent state-of-the-art two-stage networks. The source code is released in https://github.com/Mukosame/Zooming-Slow-Mo-CVPR-2020 .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑