arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2003.02438eess.IVcs.CV
收藏

利用光场的多视角透视进行弱光成像

Harnessing Multi-View Perspective of Light Fields for Low-Light Imaging

Indian Institute of Technology Madras(印度理工学院马德拉斯分校)

Mohit Lamba, Kranthi Kumar, Kaushik Mitra

摘要:

光场(LF)提供了独特的优势,如捕获后重新聚焦和深度估计,但低光条件限制了这些功能。为了恢复低光LF,我们应该利用不同LF视图中存在的几何线索,这是使用单帧低光增强技术不可能实现的。因此,我们提出了一种用于弱光光场(L3F)恢复的深度神经网络,我们称之为L3Fnet。所提出的L3Fnet不仅执行每个LF视图的必要的视觉增强,而且还保留跨视图的对极几何。我们通过采用L3Fnet的两阶段架构来实现这一点。阶段I查看所有LF视图以对LF几何结构进行编码。然后在阶段II中使用该编码信息来重构每个LF视图。为了促进基于学习的低光LF成像技术,我们收集了各种场景的全面LF数据集。对于每个场景,我们拍摄了四个LF,一个具有接近最佳的曝光和ISO设置,另一个在不同级别的低光条件下,从低光到极低光设置。建议的L3Fnet的有效性得到了该数据集上的视觉和数值比较的支持。为了进一步分析低光重建方法的性能,我们还提出了一个L3F野生数据集,其中包含在深夜捕获的LF,照度值几乎为零。此数据集中没有可用的地面实况。为了在L3F野生数据集上表现良好,任何方法都必须适应捕获场景的光照水平。为此,我们提出了一种新型预处理模块,使L3 Fnet能够在不同程度的弱光条件下保持稳健。最后,我们证明了L3Fnet也可以用于单帧图像的低光增强,尽管它是针对LF数据设计的。我们通过将单帧DSLR图像转换为适合L3Fnet的形式来实现,我们称之为伪LF。

英文摘要:

Light Field (LF) offers unique advantages such as post-capture refocusing and depth estimation, but low-light conditions limit these capabilities. To restore low-light LFs we should harness the geometric cues present in different LF views, which is not possible using single-frame low-light enhancement techniques. We, therefore, propose a deep neural network for Low-Light Light Field (L3F) restoration, which we refer to as L3Fnet. The proposed L3Fnet not only performs the necessary visual enhancement of each LF view but also preserves the epipolar geometry across views. We achieve this by adopting a two-stage architecture for L3Fnet. Stage-I looks at all the LF views to encode the LF geometry. This encoded information is then used in Stage-II to reconstruct each LF view. To facilitate learning-based techniques for low-light LF imaging, we collected a comprehensive LF dataset of various scenes. For each scene, we captured four LFs, one with near-optimal exposure and ISO settings and the others at different levels of low-light conditions varying from low to extreme low-light settings. The effectiveness of the proposed L3Fnet is supported by both visual and numerical comparisons on this dataset. To further analyze the performance of low-light reconstruction methods, we also propose an L3F-wild dataset that contains LF captured late at night with almost zero lux values. No ground truth is available in this dataset. To perform well on the L3F-wild dataset, any method must adapt to the light level of the captured scene. To do this we propose a novel pre-processing block that makes L3Fnet robust to various degrees of low-light conditions. Lastly, we show that L3Fnet can also be used for low-light enhancement of single-frame images, despite it being engineered for LF data. We do so by converting the single-frame DSLR image into a form suitable to L3Fnet, which we call as pseudo-LF.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑