Fluorescence lifetime imaging microscopy (FLIM) is an important technique to understand the chemical micro-environment in cells and tissues since it provides additional contrast compared to conventional fluorescence imaging. When two fluorophores within a diffraction limit are excited, the resulting emission leads to non-linear spatial distortion and localization effects in intensity (magnitude) and lifetime (phase) components. To address this issue, in this work, we provide a theoretical model for convolution in FLIM to describe how the resulting behavior differs from conventional fluorescence microscopy. We then present a Richardson-Lucy (RL) based deconvolution including total variation (TV) regularization method to correct for the distortions in FLIM measurements due to optical convolution, and experimentally demonstrate this FLIM deconvolution method on a multi-photon microscopy (MPM)-FLIM images of fluorescent-labeled fixed bovine pulmonary arterial endothelial (BPAE) cells.
University of Notre Dame（圣母大学）；California Institute of Technology（加州理工学院）；Indiana University（印第安纳大学）
Fluorescence lifetime imaging microscopy (FLIM) systems are limited by their slow processing speed, low signal-to-noise ratio (SNR), and expensive and challenging hardware setups. In this work, we demonstrate applying a denoising convolutional network to improve FLIM SNR. The network will be integrated with an instant FLIM system with fast data acquisition based on analog signal processing, high SNR using high-efficiency pulse-modulation, and cost-effective implementation utilizing off-the-shelf radio-frequency components. Our instant FLIM system simultaneously provides the intensity, lifetime…
Fluorescence lifetime imaging microscopy (FLIM) is a powerful technique in biomedical research that uses the fluorophore decay rate to provide additional contrast in fluorescence microscopy. However, at present, the calculation, analysis, and interpretation of FLIM is a complex, slow, and computationally expensive process. Machine learning (ML) techniques are well suited to extract and interpret measurements from multi-dimensional FLIM data sets with substantial improvement in speed over conventional methods. In this topical review, we first discuss the basics of FILM and ML. Second, we provid…
Single-sample image-fusion upsampling of fluorescence lifetime images
School of Physics & Astronomy, University of Glasgow（格拉斯哥大学物理与天文学院）；Center for Physical Sciences and Technology（物理科学与技术中心）；Cancer Research UK, Beatson Institute（英国癌症研究基金会Beatson研究所）；School of Health & Life Sciences, University of the West of Scotland（西苏格兰大学健康与生命科学学院）；University College London（伦敦大学学院）；University of Cambridge（剑桥大学）；Department of Laser Technologies（激光技术系）
In multi-view fluorescence microscopy, each angular acquisition needs to be aligned with care to obtain an optimal volumetric reconstruction. Here, instead, we propose a neat protocol based on auto-correlation inversion, that leads directly to the formation of inherently aligned tomographies. Our method generates sharp reconstructions, with the same accuracy reachable after sub-pixel alignment but with improved point-spread-function. The procedure can be performed simultaneously with deconvolution further increasing the reconstruction resolution.
University of Michigan-Shanghai Jiao Tong University Joint Institute（上海交通大学密西根学院）；Shanghai Jiao Tong University（上海交通大学）
Fluorescence microscopy plays an important role in biomedical research. The depth-variant point spread function (PSF) of a fluorescence microscope produces low-quality images especially in the out-of-focus regions of thick specimens. Traditional deconvolution to restore the out-of-focus images is usually insufficient since a depth-invariant PSF is assumed. This article aims at handling fluorescence microscopy images by learning-based depth-variant PSF and reducing artifacts. We propose adaptive weighting depth-variant deconvolution (AWDVD) with defocus level prediction convolutional neural net…
A Zero-Shot Deep Image Prior Framework for Denoising and Deconvolution in Fluorescence Microscopy
Wenzhou Key Laboratory of Biomedical Imaging（温州生物医学成像重点实验室）；Zhejiang Key Laboratory of Soft Matter Biomedical Materials（浙江软物质生物医学材料重点实验室）；Wenzhou Institute, University of Chinese Academy of Sciences（中国科学院大学温州研究院）；Postgraduate Training Base Alliance of Wenzhou Medical University（温州医学院研究生联合培养基地联盟）
UCA（蔚蓝海岸大学）；CNRS（法国国家科学研究中心）；INRIA（法国国家信息与自动化研究所）；University of Bath（巴斯大学）
The spatial resolution of images of living samples obtained by fluorescence microscopes is physically limited due to the diffraction of visible light, which makes the study of entities of size less than the diffraction barrier (around 200 nm in the x-y plane) very challenging. To overcome this limitation, several deconvolution and super-resolution techniques have been proposed. Within the framework of inverse problems, modern approaches in fluorescence microscopy reconstruct a super-resolved image from a temporal stack of frames by carefully designing suitable hand-crafted sparsity-promoting r…
Motion segmentation for natural images commonly relies on dense optic flow to yield point trajectories which can be grouped into clusters through various means including spectral clustering or minimum cost multicuts. However, in biological imaging scenarios, such as fluorescence microscopy or calcium imaging, where the signal to noise ratio is compromised and intensity fluctuations occur, optical flow may be difficult to approximate. To this end, we propose an alternative paradigm for motion segmentation based on optimal transport which models the video frames as time-varying mass represented …
Czech Academy of Sciences（捷克科学院）；Institute of Information Theory and Automation（信息理论与自动化研究所）；Washington University in St. Louis（圣路易斯华盛顿大学）；Washington University School of Medicine（华盛顿大学医学院）
Three-dimensional fluorescence microscopy often suffers from anisotropy, where the resolution along the axial direction is lower than that within the lateral imaging plane. We address this issue by presenting Dual-Cycle, a new framework for joint deconvolution and fusion of dual-view fluorescence images. Inspired by the recent Neuroclear method, Dual-Cycle is designed as a cycle-consistent generative network trained in a self-supervised fashion by combining a dual-view generator and prior-guided degradation model. We validate Dual-Cycle on both synthetic and real data showing its state-of-the-…
In single-molecule super-resolution microscopy, engineered point-spread functions (PSFs) are designed to efficiently encode new molecular properties, such as 3D orientation, into complex spatial features captured by a camera. To fully benefit from their optimality, algorithms must estimate multi-dimensional parameters such as molecular position and orientation in the presence of PSF overlap and model-experiment mismatches. Here, we present a novel joint sparse deconvolution algorithm based on the decomposition of fluorescence images into six basis images that characterize molecular orientation…
Institute of Imaging and Computer Vision, RWTH Aachen University（亚琛工业大学成像与计算机视觉研究所）
Recent developments in fluorescence microscopy allow capturing high-resolution 3D images over time for living model organisms. To be able to image even large specimens, techniques like multi-view light-sheet imaging record different orientations at each time point that can then be fused into a single high-quality volume. Based on measured point spread functions (PSF), deconvolution and content fusion are able to largely revert the inevitable degradation occurring during the imaging process. Classical multi-view deconvolution and fusion methods mainly use iterative procedures and content-based …
Spatial resolution is one of the most important specifications of an imaging system. Recent results in quantum parameter estimation theory reveal that an arbitrarily small distance between two incoherent point sources can always be efficiently determined through the use of a spatial mode sorter. However, extending this procedure to a general object consisting of many incoherent point sources remains challenging, due to the intrinsic complexity of multi-parameter estimation problems. Here, we generalize the Richardson-Lucy (RL) deconvolution algorithm to address this challenge. We simulate its …
