arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2307.02897cs.CV
收藏

RefVSR++:利用参考输入进行基于参考的视频超分辨率

RefVSR++: Exploiting Reference Inputs for Reference-based Video Super-resolution

Graduate School of Information Sciences, Tohoku University(东北大学信息科学研究生院);RIKEN Center for AIP(理化学研究所AIP中心)

Han Zou, Masanori Suganuma, Takayuki Okatani

AI总结:

本文提出RefVSR++方法,通过在时间方向上并行聚合低分辨率和参考图像并改进特征对齐机制,显著提升了基于参考的视频超分辨率性能,在PSNR上超越以往工作1dB以上。

摘要:

配备多摄像头系统、具有不同视场的智能手机正日益普及。视场的这种差异导致视频间存在内容差异,为视频超分辨率(VSR)开辟了一种创新方法。该方法通过利用高分辨率参考视频来提升低分辨率视频的VSR性能。以往基于此原理的工作通常在传统VSR模型基础上扩展,将随时间变化的LR和Ref输入合并为统一流。然而,我们预期通过独立地在时间上聚合这些Ref图像序列可以获得更好的结果。因此,我们引入了一种改进方法RefVSR++,它在时间方向上并行聚合LR和Ref图像,旨在优化可用数据的使用。RefVSR++还结合了改进的随时间对齐图像特征的机制,这对于有效的VSR至关重要。我们的实验表明,RefVSR++在PSNR上比以往工作高出1dB以上,为该领域树立了新的基准。

英文摘要:

Smartphones with multi-camera systems, featuring cameras with varying field-of-views (FoVs), are increasingly common. This variation in FoVs results in content differences across videos, paving the way for an innovative approach to video super-resolution (VSR). This method enhances the VSR performance of lower resolution (LR) videos by leveraging higher resolution reference (Ref) videos. Previous works, which operate on this principle, generally expand on traditional VSR models by combining LR and Ref inputs over time into a unified stream. However, we can expect that better results are obtained by independently aggregating these Ref image sequences temporally. Therefore, we introduce an improved method, RefVSR++, which performs the parallel aggregation of LR and Ref images in the temporal direction, aiming to optimize the use of the available data. RefVSR++ also incorporates improved mechanisms for aligning image features over time, crucial for effective VSR. Our experiments demonstrate that RefVSR++ outperforms previous works by over 1dB in PSNR, setting a new benchmark in the field.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑