基于混合镜头深度自适应融合的光场重建
Light Field Reconstruction via Deep Adaptive Fusion of Hybrid Lenses
AI总结：
本文提出一种端到端深度学习方法，通过双模块并行估计与自适应融合，从混合镜头重建高分辨率光场，在平坦区域和深度边界均优于现有方法，并降低采集成本。
摘要：
本文探讨了从混合镜头（包括一个高分辨率相机和多个环绕的低分辨率相机）重建高分辨率光场（LF）图像的问题。现有方法的性能仍然有限，因为它们在平坦纹理区域产生模糊结果，或在深度不连续边界周围产生畸变。为应对这一挑战，我们提出了一种新颖的端到端学习方法，该方法能够从两个互补且平行的视角全面利用输入的具体特征。具体而言，一个模块通过学习深度多维和跨域特征表示来回归空间一致性的中间估计，而另一个模块通过传播高分辨率视图的信息来扭曲另一个保持高频纹理的中间估计。我们最终通过学习到的注意力图自适应地利用两个中间估计的优势，从而在平坦纹理区域和深度不连续边界上都获得令人满意的最终高分辨率光场图像。此外，为了提升我们的方法在由混合光场成像系统捕获的真实混合数据上的有效性（该方法使用模拟混合数据进行训练），我们精心设计了网络架构和训练策略。在真实和模拟混合数据上的大量实验表明，我们的方法显著优于现有最先进的方法。据我们所知，这是第一个用于从真实混合输入进行光场重建的端到端深度学习方法。我们相信我们的框架有可能降低高分辨率光场数据采集的成本，并有利于光场数据的存储和传输。
英文摘要：
This paper explores the problem of reconstructing high-resolution light field (LF) images from hybrid lenses, including a high-resolution camera surrounded by multiple low-resolution cameras. The performance of existing methods is still limited, as they produce either blurry results on plain textured areas or distortions around depth discontinuous boundaries. To tackle this challenge, we propose a novel end-to-end learning-based approach, which can comprehensively utilize the specific characteristics of the input from two complementary and parallel perspectives. Specifically, one module regresses a spatially consistent intermediate estimation by learning a deep multidimensional and cross-domain feature representation, while the other module warps another intermediate estimation, which maintains the high-frequency textures, by propagating the information of the high-resolution view. We finally leverage the advantages of the two intermediate estimations adaptively via the learned attention maps, leading to the final high-resolution LF image with satisfactory results on both plain textured areas and depth discontinuous boundaries. Besides, to promote the effectiveness of our method trained with simulated hybrid data on real hybrid data captured by a hybrid LF imaging system, we carefully design the network architecture and the training strategy. Extensive experiments on both real and simulated hybrid data demonstrate the significant superiority of our approach over state-of-the-art ones. To the best of our knowledge, this is the first end-to-end deep learning method for LF reconstruction from a real hybrid input. We believe our framework could potentially decrease the cost of high-resolution LF data acquisition and benefit LF data storage and transmission.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
