Volumetric imaging of samples using fluorescence microscopy plays an important role in various fields including physical, medical and life sciences. Here we report a deep learning-based volumetric image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope at arbitrary axial positions within the sample volume. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume over an extended depth-of-field. Using experiments on C. Elegans and nanobead samples, Recurrent-MZ is demonstrated to increase the depth-of-field of a 63x/1.4NA objective lens by approximately 50-fold, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume. We further illustrated the generalization of this recurrent network for 3D imaging by showing its resilience to varying imaging conditions, including e.g., different sequences of input images, covering various axial permutations and unknown axial positioning errors. Recurrent-MZ demonstrates the first application of recurrent neural networks in microscopic image reconstruction and provides a flexible and rapid volumetric imaging framework, overcoming the limitations of current 3D scanning microscopy tools.
Fluorescence microscopy is a powerful tool for imaging biological samples with molecular specificity. In contrast, phase microscopy provides label-free measurement of the sample's refractive index (RI), which is an intrinsic optical property that quantitatively relates to cell morphology, mass, and stiffness. Conventional imaging techniques measure either the labeled fluorescence (functional) information or the label-free RI (structural) information, though it may be valuable to have both. For example, biological tissues have heterogeneous RI distributions, causing sample-induced scattering th…
We suggest a new multimodal imaging technique for quantitatively measuring the integral (thickness-average) refractive index of the nuclei of live biological cells in suspension. For this aim, we combined quantitative phase microscopy with simultaneous 2-D fluorescence microscopy. We used 2-D fluorescence microscopy to localize the nucleus inside the quantitative phase map of the cell, as well as for measuring the nucleus radii. As verified offline by both 3-D confocal fluorescence microscopy and by 2-D fluorescence microscopy while rotating the cells during flow, the nucleus of cells in suspe…
Fluorescence microscopy is widely used in biological imaging, however scattering from tissues strongly limits its applicability to a shallow depth. In this work we adapt a methodology inspired from stellar speckle interferometry, and exploit the optical memory effect to enable fluorescence microscopy through a turbid layer. We demonstrate efficient reconstruction of micrometer-size fluorescent objects behind a scattering medium in epi-microscopy, and study the specificities of this imaging modality (magnification, field of view, resolution) as compared to traditional microscopy. Using a modifi…
Super-resolution fluorescence microscopy is an important tool in biomedical research for its ability to discern features smaller than the diffraction limit. However, due to its difficult implementation and high cost, the universal application of super-resolution microscopy is not feasible. In this paper, we propose and demonstrate a new kind of super-resolution fluorescence microscopy that can be easily implemented and requires neither additional hardware nor complex post-processing. The microscopy is based on the principle of stepwise optical saturation (SOS), where $M$ steps of raw fluoresce…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California Nano Systems Institute (CNSI)（加州纳米系统研究所）；David Geffen School of Medicine（大卫·格芬医学院）；Hebrew University of Jerusalem（耶路撒冷希伯来大学）
Three-dimensional (3D) fluorescence microscopy in general requires axial scanning to capture images of a sample at different planes. Here we demonstrate that a deep convolutional neural network can be trained to virtually refocus a 2D fluorescence image onto user-defined 3D surfaces within the sample volume. With this data-driven computational microscopy framework, we imaged the neuron activity of a Caenorhabditis elegans worm in 3D using a time-sequence of fluorescence images acquired at a single focal plane, digitally increasing the depth-of-field of the microscope by 20-fold without any axi…
Fluorescence lifetime imaging microscopy (FLIM) is an important technique to understand the chemical micro-environment in cells and tissues since it provides additional contrast compared to conventional fluorescence imaging. When two fluorophores within a diffraction limit are excited, the resulting emission leads to non-linear spatial distortion and localization effects in intensity (magnitude) and lifetime (phase) components. To address this issue, in this work, we provide a theoretical model for convolution in FLIM to describe how the resulting behavior differs from conventional fluorescenc…
Breaking the diffraction limit in microscopy by utilizing quantum properties of light has been the goal of intense research in the recent years. We propose a quantum superresolution technique based on non-classical emission statistics of fluorescent markers, routinely used as contrast labels for bio-imaging. The technique can be readily implemented using standard fluorescence microscopy equipment.
Two-Photon Laser-Scanning Microscopy is a powerful tool for exploring biological structure and function because of its ability to optically section through a sample with a tight focus. While it is possible to obtain 3D image stacks by moving a stage, this perframe imaging process is time consuming. Here, we present a method for an easy to implement and inexpensive modification of an existing two-photon microscope to rapidly image in 3D using an electrically tunable lens to create a tessellating scan pattern which repeats with the volume rate. Using appropriate interpolating algorithms, the vol…
The fluorescence microscope is one of the most important tools in modern clinical diagnosis and biological science. However, its expense, size and limited field-of-view (FOV) are becoming bottlenecks in key applications such as large-scale phenotyping and low-resource-setting diagnostics. Here we report a low-cost, compact chip-scale fluorescence-imaging platform, termed the Fluorescence Talbot Microscopy (FTM), which utilizes the Talbot self-imaging effect to enable efficient fluorescence imaging over a large and directly-scalable FOV. The FTM prototype has a resolution of 1.2 microns and an …
Photobleaching is a major limitation of superresolution Stimulated Depletion Emission (STED) microscopy. Fast scanning has long been considered an effective means to reduce photobleaching in fluorescence microscopy, but a careful quantitative study of this issue is missing. In this paper, we show that the photobleaching rate in STED microscopy is slowed down and fluorescence yield is enhanced by scanning with high linear speed, enabled by the large field of view in our custom-built resonant-scanning STED microscope. The effect of scanning speed on photobleaching and fluorescence yield is more …
Nonlinear structured illumination microscopy (nSIM) is an effective approach for super-resolution wide-field fluorescence microscopy with a theoretically unlimited resolution. In nSIM, carefully designed, highly-contrasted illumination patterns are combined with the saturation of an optical transition to enable sub-diffraction imaging. While the technique proved useful for two-dimensional imaging, extending it to three-dimensions (3D) is challenging due to the fading/fatigue of organic fluorophores under intense cycling conditions. Here, we present a compressed sensing approach that allows for…
We present a new technique for obtaining simultaneous multimodal quantitative phase and fluorescence microscopy of biological cells, providing both quantitative phase imaging and molecular specificity using a single camera. Our system is based on an interferometric multiplexing module, externally positioned at the exit of an optical microscope. In contrast to previous approaches, the presented technique allows conventional fluorescence imaging, rather than interferometric off-axis fluorescence imaging. We demonstrate the presented technique for imaging fluorescent beads and live biological cel…
This Letter presents the first demonstration of multi-tile stitching for large scale 3D imaging in single objective light-sheet microscopy. We show undistorted 3D imaging spanning complete zebrafish larvae, and over 1 mm3 volumes for thick mouse brain sections. We use remote galvo scanning for light-sheet creation and develop a processing pipeline for 3D tiling across different axes. With the improved one photon (1p) tilt-invariant scanned oblique plane illumination (SOPi) microscope presented here, we demonstrate cellular resolution imaging at depths exceeding 330 um in optically scattering m…
We propose the use of the subwavelength localization via adiabatic passage technique for fluorescence microscopy with nanoscale resolution in the far field. This technique uses a Λ-type medium coherently coupled to two laser pulses: the pump, with a node in its spatial profile, and the Stokes. The population of the Λ system is adiabatically transferred from one ground state to the other except at the node position, yielding a narrow population peak. This coherent localization allows fluorescence imaging with nanometer lateral resolution. We derive an analytical expression to asses the resoluti…
Superresolution fluorescence microscopy techniques beat the diffraction limit, enabling ultra-high resolution imaging in biological physics and nanoscience. In all cases that have been studied experimentally, the resolution scales inversely with the square root of some parameter that measures the number of photons used. However, this ubiquitous limit arises from very distinct mechanisms in different approaches, raising the question of whether it is a fundamental limit that cannot be exceeded, or merely a coincidence of the techniques studied thus far. We demonstrate that, under very general as…
We build a simulation platform for Airy beam photoacoustic microscopy based on K-Wave simulation toolbox in MATLAB. The sample was located in the middle of z direction in the environment. The propagation of Airy beam irradiated on the sample was simulated by solving the Schrodinger equation. And the intensity of Airy beam was simulated by Airy wave pocket. The simulation result showed that: in A-scan, the intensity of light waved over time, reached peak at 49ns; in C-scan, the image agrees with the sample. This research showed that the Airy beam with large depth of field and no diffraction can…
Single gold nanoparticles can act as nanoantennas for enhancing the fluorescence of emitters in their near-fields. Here we present experimental and theoretical studies of scanning antenna-based fluorescence microscopy as a function of the diameter of the gold nanoparticle. We examine the interplay between fluorescence enhancement and spatial resolution and discuss the requirements for deciphering single molecules in a dense sample. Resolutions better than 20 nm and fluorescence enhancement up to 30 times are demonstrated experimentally. By accounting for the tip shaft and the sample interface …
In this letter, the leakage radiation microscopy (LRM) is extended into characterization of planar waveguide modes (WMs) rather than surface plasmon polaritons (SPPs) taking advantages of the coupling between WMs and fluorescence emission. Propagation constants of different WMs allowed in the same planar waveguide can be simultaneously and rapidly derived from the Fourier plane image of fluorescence based LRM. Numerical simulations are also carried out to calculate propagation constants of these modes, which are consistent with experimental results. Our experiments provide a simple but high ef…
We introduce the basic elements of a spatio-angular theory of fluorescence microscopy, providing a unified framework for analyzing systems that image single fluorescent dipoles and ensembles of overlapping dipoles that label biological molecules. We model an aplanatic microscope imaging an ensemble of in-focus fluorescent dipoles as a linear Hilbert-space operator, and we show that the operator takes a particularly convenient form when expressed in a basis of complex exponentials and spherical harmonics$-$a form we call the dipole spatio-angular transfer function. We discuss the implications o…
Computational Cannula Microscopy (CCM) is a high-resolution widefield fluorescence imaging approach deep inside tissue, which is minimally invasive. Rather than using conventional lenses, a surgical cannula acts as a lightpipe for both excitation and fluorescence emission, where computational methods are used for image visualization. Here, we enhance CCM with artificial neural networks to enable 3D imaging of cultured neurons and fluorescent beads, the latter inside a volumetric phantom. We experimentally demonstrate transverse resolution of ~6um, field of view ~200um and axial sectioning of ~…
In this paper we review the applicability of autofluorescent proteins for single-molecule imaging in biology. The photophysical characteristics of several mutants of the Green Fluorescent Protein (GFP) and those of DsRed are compared and critically discussed for their use in cellular biology. The alternative use of two-photon excitation at the single-molecule level or Fluorescence Correlation Spectroscopy is envisaged for the study of individual autofluorescent proteins. Single-molecule experiments performed in live cells using eGFP and preferably eYFP fusion proteins are reviewed. Finally, th…
Fluorescence imaging is the most widely used method for unveiling the molecular composition of biological specimens. However, the weak optical emission of fluorescent probes and the tradeoff between imaging speed and sensitivity is problematic for acquiring blur-free images of fast phenomena, such as sub-millisecond biochemical dynamics in live cells and tissues, and cells flowing at high speed. We report a solution that achieves real-time pixel readout rates one order of magnitude faster than a modern electron multiplier charge coupled device (EMCCD) - the gold standard in high-speed fluoresc…
FRET-based approaches are a unique tool for sensing the immediate surroundings and interactions of (bio)molecules. FRET imaging and FLIM (Fluorescence Lifetime Imaging Microscopy) enable the visualization of the spatial distribution of molecular interactions and functional states. However, conventional FLIM and FRET imaging provide average information over an ensemble of molecules within a diffraction-limited volume, which limits the spatial information, accuracy, and dynamic range of the observed signals. Here, we demonstrate an approach to obtain super-resolved FRET imaging based on single-m…
Fluorescence microscopy is indispensable in nanoscience and biological sciences. The versatility of labeling target structures with fluorescent dyes permits to visualize structure and function at a subcellular resolution with a wide field of view. Due to the diffraction limit, conventional optical microscopes are limited to resolving structures larger than 200 nm. The resolution can be enhanced by near-field and far-field super-resolution microscopy methods. Near-field methods typically have a limited field of view and far-field methods are limited by the involved conventional optics. Here, we…
Fluorescence microscopy has revolutionized biomedical research over the past three decades. Its high molecular specificity and unrivaled single molecule level sensitivity have enabled breakthroughs in a variety of research fields. For in vivo applications, its major limitation is the superficial imaging depth as random scattering in biological tissues causes exponential attenuation of the ballistic component of a light wave. Here we present fluorescence microscopy beyond the ballistic regime by combining single cycle pulsed ultrasound modulation and digital optical phase conjugation. We demons…
Amyloid proteins are associated with a broad spectrum of neurodegenerative diseases. However, it remains a grand challenge to extract molecular structure information from intracellular amyloid proteins in their native cellular environment. To address this challenge, we developed a computational chemical microscope integrating 3D mid-infrared photothermal imaging with fluorescence imaging, termed Fluorescence-guided Bond-Selective Intensity Diffraction Tomography (FBS-IDT). Based on a low-cost and simple optical design, FBS-IDT enables chemical-specific volumetric imaging and 3D site-specific m…
The optical diffraction limit, formulated by Abbe 140 years ago, imposes a bound on imaging resolution in classical optics. Over the last twenty years, many theoretical schemes have been presented for overcoming the diffraction barrier in optical imaging using quantum properties of light. An experimental realization of sub-diffraction limited quantum imaging has, however, remained elusive. Here, we demonstrate a quantum imaging method taking advantage of non-classical light naturally produced in fluorescence microscopy due to photon antibunching, a fundamentally quantum phenomenon prohibiting …
Confocal microscopy is the standard approach for obtaining volumetric images of a sample with high axial and lateral resolution, especially when dealing with scattering samples. Unfortunately, a confocal microscope is quite expensive compared to traditional microscopes. In addition, the point scanning in a confocal leads to slow imaging speed and photobleaching due to the high dose of laser energy. In this paper, we demonstrate how the advances in machine learning can be exploited to "teach" a traditional wide-field microscope, one that's available in every lab, into producing 3D volumetric im…
Super-resolution microscopy is providing unprecedented insights into biology by resolving details much below the diffraction limit. State-of-the-art Single Molecule Localization Microscopy (SMLM) techniques for super-resolution are restricted by long acquisition and computational times, or the need of special fluorophores or chemical environments. Here, we propose a novel statistical super-resolution technique of wide-field fluorescence microscopy called MUltiple SIgnal Classification ALgorithm (MUSICAL) which has several advantages over SMLM techniques. MUSICAL provides resolution down to at …
The efficient optical tool for elimination of the phase and amplitude distortions produced by imperfectness of the optical elements in microscope lightpath considered. This robust procedure described by simple theoretical model proved to be successful to repair the noisy images with noise to signal ratio close to 100. It is shown that Van Cittert - Zernike theorem provides adequate description of imperfect microscope operating under both coherent and incoherent illumination conditions and having turbid media in object and intermediate planes.
A fluorescence correlation spectroscopy (FCS) system based on two independent measurement volumes is presented. The optical setup and data acquisition hardware are detailed, as well as a complete protocol to control the location, size and shape of the measurement volumes. A method that allows to monitor independently the excitation and collection efficiency distribution is proposed. Finally, a few examples of measurements that exploit the two spots in static and/or scanning schemes, are reported.
We introduce a full field fluorescence imaging technique with axial confinement of about 100 nm at the sample/substrate interface. Contrary to standard surface imaging techniques, this confinement is obtained through emission filtering. This technique is based on supercritical emission selectivity. It can be implemented on any epifluorescence microscope with a commercial high numerical aperture objective and offers a real time surface imaging capability. This technique is of particular interest for live cell membrane and adhesion studies. Using HEK cells, we show that one can observe simultane…
Phantom eye devices are designed to mimic the optical properties of the human eye; however, the retinal structures are typically emulated with synthetic tissues. Thus, it is currently impossible to exploit phantom eye devices for fluorescence microscopy experiments on cultures, or ex-vivo tissues, although these ophthalmic measurements would be of importance for many applications such as the testing of retinal biomarkers. Here we report and describe the development of a phantom eye designed to host biological samples, such as retinal cultures differentiated from human induced pluripotent stem …
The mathematical theory of compressed sensing (CS) asserts that one can acquire signals from measurements whose rate is much lower than the total bandwidth. Whereas the CS theory is now well developed, challenges concerning hardware implementations of CS-based acquisition devices---especially in optics---have only started being addressed. This paper presents an implementation of compressive sensing in fluorescence microscopy and its applications to biomedical imaging. Our CS microscope combines a dynamic structured wide-field illumination and a fast and sensitive single-point fluorescence dete…
Imaging of rapidly occurring events in biology requires high temporal resolution. In this review article, I have tried to investigate an approximate estimate to quantify temporal resolution limit of a fluorescence imaging system. It was realised that, the temporal resolution is essentially determined by the fact that, once excited, almost all ($99.9\%$) excited molecules relax to ground state. We determined the time required for $99.9\%$ of molecules to relax is about $3τ_p = 3/{(k_f +k_{nr})\log_{10}e}$, where $k_f +k_{nr}$ is the total emission rate. To arrive at this expression we assumed t…
Scan-less full-field fluorescence-lifetime dual-comb microscopy using two-dimensional spectral mapping and frequency multiplexing of dual-optical-comb beats
We investigate the properties of a single-view fluorescence microscope in a 4$f$ geometry when imaging fluorescent dipoles without using the monopole or scalar approximations. We show that this imaging system has a spatio-angular band limit, and we exploit the band limit to perform efficient simulations. Notably, we show that information about the out-of-plane orientation of ensembles of in-focus fluorophores is recorded by paraxial fluorescence microscopes. Additionally, we show that the monopole approximation may cause biased estimates of fluorophore concentrations, but these biases are smal…
Mid-infrared photothermal microscopy is a new chemical imaging technology in which a visible beam senses the photothermal effect induced by a pulsed infrared laser. This technology provides infrared spectroscopic information at sub-micron spatial resolution and enables infrared spectroscopy and imaging of living cells and organisms. Yet, current mid-infrared photothermal imaging sensitivity suffers from a weak dependance of scattering on temperature and the image quality is vulnerable to the speckles caused by scattering. Here, we present a novel version of mid-infrared photothermal microscopy…
Optical fluorescence imaging is capable of measuring both the spatial and rotational dynamics of single molecules. However, unavoidable measurement noise will result in inaccurate estimates of rotational dynamics, causing a molecule to appear to be more rotationally constrained than it actually is. We report a mathematical framework to compute the fundamental limit of accuracy in measuring the rotational mobility of dipole-like emitters. By applying our framework to both in-plane and three-dimensional methods, we provide a means to choose the optimal orientation-measurement technique based on …
Department of Electrical Engineering, University of Notre Dame（圣母大学电气工程系）；Department of Biological Sciences, University of Notre Dame（圣母大学生物科学系）
Fluorescence microscopy has been a significant tool to observe long-term imaging of embryos (in vivo) growth over time. However, cumulative exposure is phototoxic to such sensitive live samples. While techniques like light-sheet fluorescence microscopy (LSFM) allow for reduced exposure, it is not well suited for deep imaging models. Other computational techniques are computationally expensive and often lack restoration quality. To address this challenge, one can use various low-dosage imaging techniques that are developed to achieve the 3D volume reconstruction using a few slices in the axial …
Multiphoton microscopy has enabled us to image cellular dynamics in vivo. However, the excitation wavelength for imaging with commercially available lasers is mostly limited between 650-1040 nm. Here we develop a femtosecond fiber laser system that produces $\sim$150 fs pulses at 1.8 $μ$m. Our system starts from an erbium-doped silica fiber laser, and its wavelength is converted to 1.8 $μ$m using a Raman shift fiber. The 1.8 $μ$m pulses are amplified with a two-stage Tm:ZBLAN fiber amplifier. The final pulse energy is $\sim$1 $μ$J, sufficient for in vivo imaging.
We introduce the concept of saturated absorption competition (SAC) microscopy as a means of providing sub-diffraction spatial resolution in fluorescence imaging. Unlike the post-competition process between stimulated and spontaneous emission that is used in stimulated emission depletion (STED) microscopy, SAC microscopy breaks the diffraction limit by emphasizing a pre-competition process that occurs in the fluorescence absorption stage in a manner that shares similarities with ground-state depletion (GSD) microscopy. Moreover, unlike both STED and GSD microscopy, SAC microscopy offers a reduc…
Light-field microscopy represents a promising solution for microscopic volumetric imaging, thanks to its capability to encode information on multiple planes in a single acquisition. This is achieved through its peculiar simultaneous capture of information on light spatial distribution and propagation direction. However, state-of-the-art light-field microscopes suffer from a detrimental loss of spatial resolution compared to standard microscopes. We propose and experimentally demonstrate a light-field microscopy architecture based on light intensity correlation, in which resolution is limited o…
