Conventional stereoscopic displays suffer from vergence-accommodation conflict and cause visual fatigue. Integral-imaging-based displays resolve the problem by directly projecting the sub-aperture views of a light field into the eyes using a microlens array or a similar structure. However, such displays have an inherent trade-off between angular and spatial resolutions. In this paper, we propose a novel coded time-division multiplexing technique that projects encoded sub-aperture views to the eyes of a viewer with correct cues for vergence-accommodation reflex. Given sparse light field sub-aperture views, our pipeline can provide a perception of high-resolution refocused images with minimal aliasing by jointly optimizing the sub-aperture views for display and the coded aperture pattern. This is achieved via deep learning in an end-to-end fashion by simulating light transport and image formation with Fourier optics. To our knowledge, this work is among the first that optimize the light field display pipeline with deep learning. We verify our idea with objective image quality metrics (PSNR, SSIM, and LPIPS) and perform an extensive study on various customizable design variables in our display pipeline. Experimental results show that light fields displayed using the proposed technique indeed have higher quality than that of baseline display designs.
EDoF-NeRF: extended depth-of-field neural radiance fields using a coded aperture camera
Department of Information Physics and Computing, Graduate School of Information Science and Technology, The University of Tokyo(信息物理与计算系,信息科学与技术研究生学校,东京大学)
Compressive spectral image classification using 3D coded convolutional neural network
Key Laboratory of Photoelectronic Imaging Technology and System of Ministry of Education of China, School of Optics and Photonics, Beijing Institute of Technology(北京理工大学光电学院光电成像技术与系统教育部重点实验室);Department of Electrical and Computer Engineering, University of Delaware(特拉华大学电气与计算机工程系)
Learning to Estimate Kernel Scale and Orientation of Defocus Blur with Asymmetric Coded Aperture
Tsinghua University(清华大学);Research Institute of Tsinghua University in Shenzhen(清华大学深圳研究院);Boyan Technology DBA RayShaper China(博研科技(商业名称为RayShaper中国))
Peking University Shenzhen Graduate School(北京大学深圳研究生院);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Tsinghua University(清华大学);Peking University(北京大学)
10-mega pixel snapshot compressive imaging with a hybrid coded aperture
Tsinghua University(清华大学);Massachusetts Institute of Technology(麻省理工学院);Bell Labs(贝尔实验室);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心)