Emerging single-photon-sensitive sensors combined with advanced inverse methods to process picosecond-accurate time-stamped photon counts have given rise to unprecedented imaging capabilities. Rather than imaging photons that travel along direct paths from a source to an object and back to the detector, non-line-of-sight (NLOS) imaging approaches analyse photons {scattered from multiple surfaces that travel} along indirect light paths to estimate 3D images of scenes outside the direct line of sight of a camera, hidden by a wall or other obstacles. Here we review recent advances in the field of NLOS imaging, discussing how to see around corners and future prospects for the field.
We develop a scannerless non-line-of-sight three dimensional imaging system based on a commercial 32x32 SPAD camera combined with a 70 ps pulsed laser. In our experiment, 1024 time histograms can be achieved synchronously in 3s with an average time resolution of about 165 ps. The result with filtered back projection shows a discernable reconstruction while the result using virtual wave field demonstrates a better quality similar to the ones created by earlier scanning imaging systems with single pixel SPAD. Comparatively, our system has large potential advantages in frame frequency, power requ…
Real time, high resolution 3D reconstruction of scenes hidden from the direct field of view is a challenging field of research with applications in real-life situations related e.g. to surveillance, self-driving cars and rescue missions. Most current techniques recover the 3D structure of a non-lineof-sight (NLOS) static scene by detecting the return signal from the hidden object on a scattering observation area. Here, we demonstrate the full colour retrieval of the 3D shape of a hidden scene by coupling back-projection imaging algorithms with the high-resolution time-of-flight information pro…
Non-line-of-sight (NLOS) imaging relies on collecting light that is rendered incoherent from the multiple scattering events and is then post-processed to provide an estimate of the hidden scene. Here we employ coherent phase control of the outgoing laser beam phase front so as to refocus the beam behind the obscuring obstacle and then use the speckle memory effect to scan the focused spot across the scene. The back-reflected light intensity provides a direct measurement of the scene with a signal-to-noise ratio that is greatly improved when measured using a temporally gated detector. A spatial…
Non-line-of-sight (NLOS) imaging has recently attracted a lot of interest from the scientific community. The goal of this paper is to provide the basis for a comprehensive mathematical framework for NLOS imaging that is directly derived from physical concepts. We introduce the irradiance phasor field (P-field) as an abstract quantity for irradiance fluctuations, akin to the complex envelope of the Electrical field (E-field) that is used to describe propagation of electromagnetic energy. We demonstrate that the P-field propagator is analogous to the Huygens-Fresnel propagator that describes the…
Non-line-of-sight (NLOS) imaging enables monitoring around corners and is promising for diverse applications. The resolution of transient NLOS imaging is limited to a centimeter scale, mainly by the temporal resolution of the detectors. Here, we construct an up-conversion single-photon detector with a high temporal resolution of ~1.4 ps and a low noise count rate of 5 counts per second (cps). Notably, the detector operates at room temperature, near-infrared wavelength. Using this detector, we demonstrate high-resolution and low-noise NLOS imaging. Our system can provide a 180 μm axial resoluti…
A deep learning based non-line-of-sight (NLOS) imaging system is developed to image an occluded object off a scattering surface. The neural net is trained using only handwritten digits, and yet exhibits capability to reconstruct patterns distinct from the training set, including physical objects. It can also reconstruct a cartoon video from its scattering patterns in real time, demonstrating the robustness and generalization capability of the deep learning based approach. Several scattering surfaces with varying degree of Lambertian and specular contributions were examined experimentally; it i…
Non-line-of-sight (NLOS) imaging techniques use light that diffusely reflects off of visible surfaces (e.g., walls) to see around corners. One approach involves using pulsed lasers and ultrafast sensors to measure the travel time of multiply scattered light. Unlike existing NLOS techniques that generally require densely raster scanning points across the entirety of a relay wall, we explore a more efficient form of NLOS scanning that reduces both acquisition times and computational requirements. We propose a circular and confocal non-line-of-sight (C2NLOS) scan that involves illuminating and im…
Non-line-of-sight imaging with arbitrary relay surface geometries via 3D Gaussian Transient Rendering
Department of Precise Instrument, Tsinghua University（清华大学精密仪器系）；Department of Electrical Engineering, City University of Hong Kong（香港城市大学电子工程系）；Yau Mathematical Science Center, Tsinghua University（清华大学姚期智科学中心）
Tsinghua University（清华大学）；Yanqi Lake Beijing Institute of Mathematical Sciences and Applications（雁栖湖北京数学科学与应用研究院）
The non-line-of-sight imaging technique aims to reconstruct targets from multiply reflected light. For most existing methods, dense points on the relay surface are raster scanned to obtain high-quality reconstructions, which requires a long acquisition time. In this work, we propose a signal-surface collaborative regularization (SSCR) framework that provides noise-robust reconstructions with a minimal number of measurements. Using Bayesian inference, we design joint regularizations of the estimated signal, the 3D voxel-based representation of the objects, and the 2D surface-based description o…
We consider the non-line-of-sight (NLOS) imaging of an object using the light reflected off a diffusive wall. The wall scatters incident light such that a lens is no longer useful to form an image. Instead, we exploit the 4D spatial coherence function to reconstruct a 2D projection of the obscured object. The approach is completely passive in the sense that no control over the light illuminating the object is assumed and is compatible with the partially coherent fields ubiquitous in both the indoor and outdoor environments. We formulate a multi-criteria convex optimization problem for reconstr…
Visual object recognition is one of the most important perception functions for a wide range of intelligent machines. A conventional recognition process begins with forming a clear optical image of the object, followed by its computer analysis. In contrast, it is possible to carry out recognition without imaging by using coherent illumination and directly analyzing the optical interference pattern of the scattered light as captured by an image sensor. Here we show that such direct visual recognition can overcome traditional limitations of imaging optics to realize excellent recognition without…
Non-line-of-sight imaging has attracted more attentions for its wide applications.Even though ultrasensitive cameras or detectors with high time-resolution are available, current back-projection methods are still powerless to acquire a satisfying reconstruction of multiple hidden objects due to severe aliasing artifacts. Here, a novel back-projection method is developed to reconstruct multiple hidden objects. Our method considers decomposing all the ellipsoids in a confidence map into several clusters belonging to different objects (namely ellipsoid mode decomposition), and then reconstructing…
Coherent AI LLC（Coherent AI有限责任公司）；Huazhong University of Science and Technology（华中科技大学）；Stanford University（斯坦福大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Visual object recognition under situations in which the direct line-of-sight is blocked, such as when it is occluded around the corner, is of practical importance in a wide range of applications. With coherent illumination, the light scattered from diffusive walls forms speckle patterns that contain information of the hidden object. It is possible to realize non-line-of-sight (NLOS) recognition with these speckle patterns. We introduce a novel approach based on speckle pattern recognition with deep neural network, which is simpler and more robust than other NLOS recognition methods. Simulation…
Optical Coherence Tomography (OCT) is a key 3D imaging technology that provides micron scale depth resolution for bio-imaging. This resolution substantially surpasses what it typically achieved in Light Detection and Ranging (LiDAR) which is often limited to the millimetre scale due to the impulse response of the detection electronics. However, the lack of coherence in LiDAR scenes, arising from mechanical motion for example, make OCT practically infeasible. Here we present a quantum interference inspired approach to LiDAR which achieves OCT depth resolutions without the need for high levels o…
