arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.08058cs.CV
收藏

学习光场图像超分辨率的非局部空间-角度相关性

Learning Non-Local Spatial-Angular Correlation for Light Field Image Super-Resolution

National University of Defense Technology(国防科技大学);Aviation University of Air Force(空军航空大学)

Zhengyu Liang, Yingqian Wang, Longguang Wang, Jungang Yang, Shilin Zhou, Yulan Guo

AI总结:

本文提出利用EPI表示和Transformer自注意力学习光场图像的非局部空间-角度相关性,在五个公开数据集上取得最先进超分辨率性能并对视差变化具有鲁棒性。

摘要:

利用空间-角度相关性对于光场(LF)图像超分辨率(SR)至关重要,但由于LF图像之间的视差所导致的非局部特性,这一任务极具挑战性。尽管许多深度神经网络(DNN)已被开发用于LF图像SR并持续取得性能提升,但现有方法无法充分利用长程空间-角度相关性,因此在处理视差变化较大的场景时会出现显著的性能下降。本文提出了一种简单而有效的方法来学习用于LF图像SR的非局部空间-角度相关性。在该方法中,我们采用极线平面图像(EPI)表示将4D空间-角度相关性投影到多个2D EPI平面上,然后开发了一个具有重复自注意力操作的Transformer网络,通过建模每对EPI像素之间的依赖关系来学习空间-角度相关性。我们的方法能够充分整合来自所有角度视图的信息,同时沿极线获得全局感受野。我们进行了大量实验并辅以有洞察力的可视化来验证所提方法的有效性。在五个公开数据集上的对比结果表明,我们的方法不仅实现了最先进的SR性能,而且对视差变化具有鲁棒性。代码已公开于 https://github.com/ZhengyuLiang24/EPIT。

英文摘要:

Exploiting spatial-angular correlation is crucial to light field (LF) image super-resolution (SR), but is highly challenging due to its non-local property caused by the disparities among LF images. Although many deep neural networks (DNNs) have been developed for LF image SR and achieved continuously improved performance, existing methods cannot well leverage the long-range spatial-angular correlation and thus suffer a significant performance drop when handling scenes with large disparity variations. In this paper, we propose a simple yet effective method to learn the non-local spatial-angular correlation for LF image SR. In our method, we adopt the epipolar plane image (EPI) representation to project the 4D spatial-angular correlation onto multiple 2D EPI planes, and then develop a Transformer network with repetitive self-attention operations to learn the spatial-angular correlation by modeling the dependencies between each pair of EPI pixels. Our method can fully incorporate the information from all angular views while achieving a global receptive field along the epipolar line. We conduct extensive experiments with insightful visualizations to validate the effectiveness of our method. Comparative results on five public datasets show that our method not only achieves state-of-the-art SR performance, but also performs robust to disparity variations. Code is publicly available at https://github.com/ZhengyuLiang24/EPIT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑