基于几何感知网络的光场角超分辨率学习
Learning Light Field Angular Super-Resolution via a Geometry-Aware Network
摘要:
获取具有高角分辨率的光场图像是昂贵的。虽然已经提出了许多方法来提高稀疏采样光场的角分辨率,但是它们总是集中在由消费者光场相机捕获的具有小基线的光场。通过充分利用光场固有的\textit{geometry}信息,提出了一种基于端到端学习的大基线稀疏光场角度超分辨方法.我们的模型由两个可学习模块和一个基于物理的模块组成。具体地,它包括用于明确地对场景几何形状建模的深度估计模块、用于新颖视图合成的基于物理的扭曲、以及专门设计用于光场重构的光场混合模块。此外,我们引入了一种新的损失函数,以促进光场视差结构的保存。在包括大基线光场图像的各种光场数据集上的实验结果表明,与现有技术相比,我们的方法具有显著的优越性,即,我们的方法将第二好方法的PSNR平均提高了2 dB,同时节省了48 $\times$的执行时间。此外,我们的方法更好地保留了光场视差结构。
英文摘要:
The acquisition of light field images with high angular resolution is costly. Although many methods have been proposed to improve the angular resolution of a sparsely-sampled light field, they always focus on the light field with a small baseline, which is captured by a consumer light field camera. By making full use of the intrinsic \textit{geometry} information of light fields, in this paper we propose an end-to-end learning-based approach aiming at angularly super-resolving a sparsely-sampled light field with a large baseline. Our model consists of two learnable modules and a physically-based module. Specifically, it includes a depth estimation module for explicitly modeling the scene geometry, a physically-based warping for novel views synthesis, and a light field blending module specifically designed for light field reconstruction. Moreover, we introduce a novel loss function to promote the preservation of the light field parallax structure. Experimental results over various light field datasets including large baseline light field images demonstrate the significant superiority of our method when compared with state-of-the-art ones, i.e., our method improves the PSNR of the second best method up to 2 dB in average, while saves the execution time 48$\times$. In addition, our method preserves the light field parallax structure better.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。