基于灵活采样和几何感知融合的深度由粗到细密集光场重建
Deep Coarse-to-fine Dense Light Field Reconstruction with Flexible Sampling and Geometry-aware Fusion
摘要：
密集采样的光场（LF）在各种应用中是高度期望的，诸如3-D重建、捕获后重聚焦和虚拟现实。然而，获取这些数据的成本很高。尽管已经提出了许多计算方法来从稀疏采样的LF重构密集采样的LF，但是它们仍然遭受低重构质量、低计算效率或者对采样模式的规则性的限制。为此，我们提出了一种新的基于学习的方法，该方法接受具有不规则结构的稀疏采样LF，并准确有效地产生具有任意角分辨率的密集采样LF。我们还提出了一个简单而有效的方法来优化采样模式。我们提出的方法是一种端到端的可训练网络，以粗到细的方式重建密集采样的LF。具体而言，粗子孔径图像（SAI）合成模块首先从非结构化稀疏采样LF探索场景几何形状，并利用它独立地合成新的SAI，其中提出了一种基于置信度的混合策略来融合来自不同输入SAI的信息，给出中间密集采样LF。然后，高效LF细化模块学习中间结果内的角度关系以恢复LF视差结构。综合实验评估表明，我们的方法在现实世界和合成LF图像相比，国家的最先进的方法的优越性。此外，我们说明了所提出的方法的好处和优势时，应用在各种基于LF的应用程序，包括基于图像的渲染和深度估计增强。
英文摘要：
A densely-sampled light field (LF) is highly desirable in various applications, such as 3-D reconstruction, post-capture refocusing and virtual reality. However, it is costly to acquire such data. Although many computational methods have been proposed to reconstruct a densely-sampled LF from a sparsely-sampled one, they still suffer from either low reconstruction quality, low computational efficiency, or the restriction on the regularity of the sampling pattern. To this end, we propose a novel learning-based method, which accepts sparsely-sampled LFs with irregular structures, and produces densely-sampled LFs with arbitrary angular resolution accurately and efficiently. We also propose a simple yet effective method for optimizing the sampling pattern. Our proposed method, an end-to-end trainable network, reconstructs a densely-sampled LF in a coarse-to-fine manner. Specifically, the coarse sub-aperture image (SAI) synthesis module first explores the scene geometry from an unstructured sparsely-sampled LF and leverages it to independently synthesize novel SAIs, in which a confidence-based blending strategy is proposed to fuse the information from different input SAIs, giving an intermediate densely-sampled LF. Then, the efficient LF refinement module learns the angular relationship within the intermediate result to recover the LF parallax structure. Comprehensive experimental evaluations demonstrate the superiority of our method on both real-world and synthetic LF images when compared with state-of-the-art methods. In addition, we illustrate the benefits and advantages of the proposed approach when applied in various LF-based applications, including image-based rendering and depth estimation enhancement.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
