混合透镜的注意力引导融合实现光场超分辨率
Light Field Super-resolution via Attention-Guided Fusion of Hybrid Lenses
摘要：
本文研究了由混合透镜重建高分辨率光场图像的问题，包括由多个低分辨率相机包围的高分辨率相机。为了应对这一挑战，我们提出了一种新的基于端到端学习的方法，该方法可以从两个互补和并行的角度综合利用输入的具体特征。具体而言，一个模块通过学习深度多维和跨域特征表示来回归空间一致的中间估计;另一个模块通过传播高分辨率视图的信息来构建另一个中间估计，该中间估计保持高频纹理。最后，我们通过学习的注意力地图利用两个中间估计的优势，从而得到最终的高分辨率LF图像。大量的实验表明，我们的方法比国家的最先进的显着优越性。也就是说，我们的方法不仅将PSNR提高了2 dB以上，而且更好地保留了LF结构。据我们所知，这是第一个端到端的深度学习方法，用于重建具有混合输入的高分辨率LF图像。我们相信我们的框架可以潜在地降低高分辨率LF数据采集的成本，也有利于LF数据的存储和传输。该代码可在https://github.com/jingjin25/LFhybridSR-Fusion上获得。
英文摘要：
This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses, including a high-resolution camera surrounded by multiple low-resolution cameras. To tackle this challenge, we propose a novel end-to-end learning-based approach, which can comprehensively utilize the specific characteristics of the input from two complementary and parallel perspectives. Specifically, one module regresses a spatially consistent intermediate estimation by learning a deep multidimensional and cross-domain feature representation; the other one constructs another intermediate estimation, which maintains the high-frequency textures, by propagating the information of the high-resolution view. We finally leverage the advantages of the two intermediate estimations via the learned attention maps, leading to the final high-resolution LF image. Extensive experiments demonstrate the significant superiority of our approach over state-of-the-art ones. That is, our method not only improves the PSNR by more than 2 dB, but also preserves the LF structure much better. To the best of our knowledge, this is the first end-to-end deep learning method for reconstructing a high-resolution LF image with a hybrid input. We believe our framework could potentially decrease the cost of high-resolution LF data acquisition and also be beneficial to LF data storage and transmission. The code is available at https://github.com/jingjin25/LFhybridSR-Fusion.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
