arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1802.01722cs.CV
收藏

使用深度学习的压缩光场重建

Compressive Light Field Reconstructions using Deep Learning

Arizona State University(亚利桑那州立大学);Cornell University(康奈尔大学);Carnegie Mellon University(卡内基梅隆大学)

Mayank Gupta, Arjun Jauhari, Kuldeep Kulkarni, Suren Jayasuriya, Alyosha Molnar, Pavan Turaga

摘要:

光场成像在其空间和角度维度的高采样的计算处理要求方面受到限制。单次光场相机牺牲空间分辨率来采样角度视点,通常通过将入射光线多路复用到2D传感器阵列上。虽然可以使用压缩感测来恢复该分辨率,但是这些迭代解决方案在处理光场时是缓慢的。我们提出了一种深度学习方法,使用一种新的两分支网络架构,由一个自动编码器和一个4D CNN共同组成,从一个编码的2D图像中恢复一个高分辨率的4D光场。该网络显著减少了重建时间,同时在各种光场上实现了26-32 dB的平均PSNR值。特别是,重建时间从35分钟减少到6.7分钟相比,字典的方法相同的视觉质量。这些重建以低至8%的小采样/压缩比执行,从而允许更便宜的编码光场相机。我们测试我们的网络重建的合成光场,从Lytro Illum相机捕获的真实光场的模拟编码测量,以及来自自定义CMOS衍射光场相机的真实编码图像。压缩光场捕获与深度学习的组合允许未来的实时光场视频采集系统的潜力。

英文摘要:

Light field imaging is limited in its computational processing demands of high sampling for both spatial and angular dimensions. Single-shot light field cameras sacrifice spatial resolution to sample angular viewpoints, typically by multiplexing incoming rays onto a 2D sensor array. While this resolution can be recovered using compressive sensing, these iterative solutions are slow in processing a light field. We present a deep learning approach using a new, two branch network architecture, consisting jointly of an autoencoder and a 4D CNN, to recover a high resolution 4D light field from a single coded 2D image. This network decreases reconstruction time significantly while achieving average PSNR values of 26-32 dB on a variety of light fields. In particular, reconstruction time is decreased from 35 minutes to 6.7 minutes as compared to the dictionary method for equivalent visual quality. These reconstructions are performed at small sampling/compression ratios as low as 8%, allowing for cheaper coded light field cameras. We test our network reconstructions on synthetic light fields, simulated coded measurements of real light fields captured from a Lytro Illum camera, and real coded images from a custom CMOS diffractive light field camera. The combination of compressive light field capture with deep learning allows the potential for real-time light field video acquisition systems in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑