Structured illumination microscopy (SIM) can double the resolution beyond the light diffraction limit, but it comes at the cost of multiple camera exposures and the heavy computation burden of multiple Fourier transforms. In this paper, we report a novel technique termed single-shot SIM, to overcome these limitations. A multi-task joint deep-learning strategy is proposed. Generative adversative networks (GAN) are employed to generate five structured illumination images based on the single-shot structured illumination image. U-Net is employed to reconstruct the super-resolution image from these six generated images without time-consuming Fourier transform. By imaging a self-assembling DNB array, we experimentally verified that this technique could perform single-shot super-resolution reconstruction comparing favorably with conventional SIM. This single-shot SIM technique may ultimately overcome the limitations of multiple exposures and Fourier transforms and is potentially applied for high-throughput gene sequencing.
High-content biological microscopy targets high-resolution imaging across large fields-of-view (FOVs). Recent works have demonstrated that computational imaging can provide efficient solutions for high-content microscopy. Here, we use speckle structured illumination microscopy (SIM) as a robust and cost-effective solution for high-content fluorescence microscopy with simultaneous high-content quantitative phase (QP). This multi-modal compatibility is essential for studies requiring cross-correlative biological analysis. Our method uses laterally-translated Scotch tape to generate high-resoluti…
A method is proposed for assessing the temporal resolution of Structured Illumination Microscopy (SIM), by tracking the amplitude of different spatial frequency components over time, and comparing them to a temporally-oscillating ground-truth. This method is used to gain insight into the performance limits of SIM, along with alternative reconstruction techniques (termed 'rolling SIM') that claim to improve temporal resolution. Results show that the temporal resolution of SIM varies considerably between low and high spatial frequencies, and that, despite being used in several high profile paper…
Structured Illumination Microscopy (SIM) is an imaging technique for achieving both super-resolution (SR) and optical sectioning (OS) in wide-field microscopy. It consists in illuminating the sample with periodic patterns at different orientations and positions. The resulting images are then processed to reconstruct the observed object with SR and/or OS. In this work, we present BOSSA-SIM, a general-purpose SIM reconstruction method, applicable to moving objects such as encountered in in vivo retinal imaging, that enables SR and OS jointly in a fully unsupervised Bayesian framework. By modelin…
Structured illumination microscopy (SIM) achieves doubled spatial resolution by exciting the specimen with a high-contrast, high-frequency sinusoidal pattern. Such an excitation pattern can be generated by interference between multiple laser beams, which are diffracted from a grating. In SIM, 2D imaging requires 9 patterns and 3D imaging requires 15 patterns. Compared to mechanical movement of gratings, opti-electro devices provide rapid switch of the excitation patterns, in which Digital Micro-mirror Device (DMD) is most common in industry. Here we model DMD as the blazed grating and report a…
Fourier ptychographic microscopy allows for the collection of images with a high space-bandwidth product at the cost of temporal resolution. In Fourier ptychographic microscopy, the light source of a conventional widefield microscope is replaced with a light-emitting diode (LED) matrix, and multiple images are collected with different LED illumination patterns. From these images, a higher-resolution image can be computationally reconstructed without sacrificing field-of-view. We use deep learning to achieve single-shot imaging without sacrificing the space-bandwidth product, reducing the acqui…
Digital two-dimensional (2D) spatial sampling devices (such as charge-coupled device) have been widely used in various imaging systems, especially in computational imaging systems. However, the undersampling of digital sampling devices is a problem that limits the resolution of the acquired images. In this study, we present a synthetic sampling imaging (SSI) concept to solve the undersampling problem. It combines the structured illumination system and conventional 2D image detection system to simultaneously sample the specimen from the illumination and the detection sides. Then, we synthesize …
The structured illumination microscopy using unknown speckle patterns has shown the capacity to surpass the Abbe's diffraction barrier, giving the possibility to design cheap and versatile SIM devices. However, the state-of-the-art joint reconstruction methods in this framework has a relatively low contrast in super-resolution part in comparison to conventional SIM and the hyper-parameter is not easy to tune. In this paper, a unified joint reconstruction approach is proposed with the hyper-parameter proportional to the noise level. Different regularization terms could be evaluated under the sa…
The transport-of-intensity equation (TIE) is a well-established non-interferometric phase retrieval approach, which enables quantitative phase imaging (QPI) of transparent sample simply by measuring the intensities at multiple axially displaced planes. Nevertheless, it still suffers from two fundamentally limitations. First, it is quite susceptible to low-frequency errors (such as \cloudy" artifacts), which results from the poor contrast of the phase transfer function (PTF) near the zero frequency. Second, the reconstructed phase tends to blur under spatially low-coherent illumination, especia…
Conventional approaches for 3D imaging in or through scattering media are usually limited to 2D reconstruction of objects at some discontinuous locations, although the time-consuming iteration, guide-star, or complex system are implemented. How to quickly visualize dynamic 3D objects behind scattering media is still an open issue. Here, by using structured light illumination, we propose a single-shot technique that can quickly acquire continuous 3D surfaces of objects hidden behind the diffuser. The proposed method can realize the 3D imaging of single, multiple, and dynamic targets from the sp…
In recent years there has been great interest in using deep neural networks (DNN) for super-resolution image reconstruction including for structured illumination microscopy (SIM). While these methods have shown very promising results, they all rely on data-driven, supervised training strategies that need a large number of ground truth images, which is experimentally difficult to realize. For SIM imaging, there exists a need for a flexible, general, and open-source reconstruction method that can be readily adapted to different forms of structured illumination. We demonstrate that we can combine…
Many optical microscopy techniques rely on structured illumination by way of a projected image (eg. structured illumination microscopy) or a tailored angular distribution of light (eg. ptychography). Electro-optical equipment such as spatial light modulators and light emitting diode (LED) arrays are commonly used to sculpt light for these imaging schemes. However, these high-tech devices are not a requirement for light crafting. Patterned diffusely reflecting surfaces also imprint a spatio-angular structure onto reflected light. We demonstrate that paper patterned with a standard commercial pr…
The performance of light-field microscopy is improved by selectively illuminating the relevant subvolume of the specimen with a second objective lens [1-3]. Here we advance this approach to a single-objective geometry, using an oblique one-photon illumination path or two-photon illumination to accomplish selective-volume excitation. The elimination of the second orthogonally oriented objective to selectively excite the volume of interest simplifies specimen mounting; yet, this single-objective approach still reduces out-of-volume background, resulting in improvements in image contrast, effecti…
Fourier ptychographic microscopy (FPM) can achieve quantitative phase imaging with a large space-bandwidth product by synthesizing a set of low-resolution intensity images captured under angularly varying illuminations. Determining accurate illumination angles is critical because the consistency between actual systematic parameters and those used in the recovery algorithm is essential for high-quality imaging. This paper presents a full-pose-parameter and physics-based method for calibrating illumination angles. Using a physics-based model constructed with general knowledge of the employed mic…
Nanjing University of Science and Technology（南京理工大学）；Soochow University（苏州大学）
Structured illumination microscopy (SIM) is an important super-resolution based microscopy technique that breaks the diffraction limit and enhances optical microscopy systems. With the development of biology and medical engineering, there is a high demand for real-time and robust SIM imaging under extreme low light and short exposure environments. Existing SIM techniques typically require multiple structured illumination frames to produce a high-resolution image. In this paper, we propose a single-frame structured illumination microscopy (SF-SIM) based on deep learning. Our SF-SIM only needs o…
This work presents the development of a flexible microscopic chemical imaging platform for cultural heritage that utilizes wavelength-tunable oblique illumination from a point source to obtain per-pixel reflectance spectra in the VIS-NIR range. The microscope light source can be adjusted on two axes allowing for a hemisphere of possible illumination directions. The synthesis of multiple illumination angles allows for the calculation of surface normal vectors, similar to phase gradients, and axial optical sectioning. The extraction of spectral reflectance images with high spatial resolutions fr…
Deconvolution phase microscopy enables high-contrast visualization of transparent samples through reconstructions of their transmitted phases or refractive indexes. Herein, we propose a method to extend 2D deconvolution phase microscopy to thick 3D samples. The refractive index distribution of a sample can be obtained at a specific axial plane by measuring only four intensity images obtained under optimized illumination patterns. Also, the optical phase delay of a sample can be measured using different illumination patterns.
University of Cambridge（剑桥大学）；Department of Chemical Engineering and Biotechnology（化学工程与生物技术系）；Department of Computer Science and Technology（计算机科学与技术系）
Structured illumination microscopy (SIM) has become an important technique for optical super-resolution imaging because it allows a doubling of image resolution at speeds compatible for live-cell imaging. However, the reconstruction of SIM images is often slow and prone to artefacts. Here we propose a versatile reconstruction method, ML-SIM, which makes use of machine learning. The model is an end-to-end deep residual neural network that is trained on a simulated data set to be free of common SIM artefacts. ML-SIM is thus robust to noise and irregularities in the illumination patterns of the r…
White light phase-shifting interference microscopy (WL-PSIM) is a prominent technique for high-resolution quantitative phase imaging (QPI) of industrial and biological specimens. However, multiple interferograms with accurate phase-shifts are essentially required in WL-PSIM for measuring the accurate phase of the object. Here, we present single-shot phase-shifting interferometric techniques for accurate phase measurement using filtered white light phase-shifting interference microscopy (F-WL-PSIM) and deep neural network (DNN). The methods are incorporated by training the DNN to generate 1) fo…
We report a compact, cost-effective and field-portable lensless imaging platform for quantitative microscopy. In this platform, the object is placed on top of an image sensor chip without using any lens. We use a low-cost galvo scanner to rapidly scan an unknown laser speckle pattern on the object. To address the positioning repeatability and accuracy issues, we directly recover the positional shifts of the speckle pattern based on the phase correlation of the captured images. To bypass the resolution limit set by the imager pixel size, we employ a sub-sampled ptychographic phase retrieval pro…
Fourier ptychography microscopy (FPM), sharing its roots with synthetic aperture technique and phase retrieval method, is a recently developed computational microscopic super-resolution technique. By turning on the light-emitting diode (LED) elements sequentially and acquiring the corresponding images that contain different spatial frequencies, FPM can achieve a wide field-of-view (FOV), high-spatial-resolution imaging, and phase recovery simultaneously. Conventional FPM assumes that the sample is sufficiently thin and strictly in focus. Nevertheless, even for a relatively thin sample, the non…
This paper introduces a new method of data-driven microscope design for virtual fluorescence microscopy. Our results show that by including a model of illumination within the first layers of a deep convolutional neural network, it is possible to learn task-specific LED patterns that substantially improve the ability to infer fluorescence image information from unstained transmission microscopy images. We validated our method on two different experimental setups, with different magnifications and different sample types, to show a consistent improvement in performance as compared to conventional…
We demonstrate a deep learning-based offline autofocusing method, termed Deep-R, that is trained to rapidly and blindly autofocus a single-shot microscopy image of a specimen that is acquired at an arbitrary out-of-focus plane. We illustrate the efficacy of Deep-R using various tissue sections that were imaged using fluorescence and brightfield microscopy modalities and demonstrate snapshot autofocusing under different scenarios, such as a uniform axial defocus as well as a sample tilt within the field-of-view. Our results reveal that Deep-R is significantly faster when compared with standard …
The trade-off between throughput and image quality is an inherent challenge in microscopy. To improve throughput, compressive imaging under-samples image signals; the images are then computationally reconstructed by solving a regularized inverse problem. Compared to traditional regularizers, Deep Learning based methods have achieved greater success in compression and image quality. However, the information loss in the acquisition process sets the compression bounds. Further improvement in compression, without compromising the reconstruction quality is thus a challenge. In this work, we propose…
Background: The use of whole slide imaging (WSI) for digital pathology has recently been cleared for primary diagnosis in the US. A conventional WSI system scans the tissue slide to different positions and acquires the digital images. In a typical implementation, a focus map is created prior to the scanning process, leading to significant overhead time and a necessity for high positional accuracy of the mechanical system. The resulting cost of WSI system is often prohibitive for frozen section procedure during surgery. Methods: We report a novel WSI scheme based on a programmable LED array for…
