Single molecule fluorescence microscopy is a powerful technique for uncovering detailed information about biological systems, both in vitro and in vivo. In such experiments, the inherently low signal to noise ratios mean that accurate algorithms to separate true signal and background noise are essential to generate meaningful results. To this end, we have developed a new and robust method to reduce noise in single molecule fluorescence images by using a Gaussian Markov Random Field (GMRF) prior in a Bayesian framework. Two different strategies are proposed to build the prior - an intrinsic GMRF, with a stationary relationship between pixels and a heterogeneous intrinsic GMRF, with a differently weighted relationship between pixels classified as molecules and background. Testing with synthetic and real experimental fluorescence images demonstrates that the heterogeneous intrinsic GMRF is superior to other conventional de-noising approaches.
Fluorescence microscopy has enabled a dramatic development in modern biology. Due to its inherently weak signal, fluorescence microscopy is not only much noisier than photography, but also presented with Poisson-Gaussian noise where Poisson noise, or shot noise, is the dominating noise source. To get clean fluorescence microscopy images, it is highly desirable to have effective denoising algorithms and datasets that are specifically designed to denoise fluorescence microscopy images. While such algorithms exist, no such datasets are available. In this paper, we fill this gap by constructing a …
Northwestern University（西北大学）；Delft University of Technology（代尔夫特理工大学）
Macro x-ray fluorescence (XRF) imaging of cultural heritage objects, while a popular non-invasive technique for providing elemental distribution maps, is a slow acquisition process in acquiring high signal-to-noise ratio XRF volumes. Typically on the order of tenths of a second per pixel, a raster scanning probe counts the number of photons at different energies emitted by the object under x-ray illumination. In an effort to reduce the scan times without sacrificing elemental map and XRF volume quality, we propose using dictionary learning with a Poisson noise model as well as a color image-ba…
Fluorescence microscopy is a key driver to promote discoveries of biomedical research. However, with the limitation of microscope hardware and characteristics of the observed samples, the fluorescence microscopy images are susceptible to noise. Recently, a few self-supervised deep learning (DL) denoising methods have been proposed. However, the training efficiency and denoising performance of existing methods are relatively low in real scene noise removal. To address this issue, this paper proposed self-supervised image denoising method Noise2SR (N2SR) to train a simple and effective image den…
University of Notre Dame（圣母大学）；California Institute of Technology（加州理工学院）；Indiana University（印第安纳大学）
Fluorescence lifetime imaging microscopy (FLIM) systems are limited by their slow processing speed, low signal-to-noise ratio (SNR), and expensive and challenging hardware setups. In this work, we demonstrate applying a denoising convolutional network to improve FLIM SNR. The network will be integrated with an instant FLIM system with fast data acquisition based on analog signal processing, high SNR using high-efficiency pulse-modulation, and cost-effective implementation utilizing off-the-shelf radio-frequency components. Our instant FLIM system simultaneously provides the intensity, lifetime…
Due to image blurring image deconvolution is often used for studying biological structures in fluorescence microscopy. Fluorescence microscopy image volumes inherently suffer from intensity inhomogeneity, blur, and are corrupted by various types of noise which exacerbate image quality at deeper tissue depth. Therefore, quantitative analysis of fluorescence microscopy in deeper tissue still remains a challenge. This paper presents a three dimensional blind image deconvolution method for fluorescence microscopy using 3-way spatially constrained cycle-consistent adversarial networks. The restored…
A Zero-Shot Deep Image Prior Framework for Denoising and Deconvolution in Fluorescence Microscopy
Wenzhou Key Laboratory of Biomedical Imaging（温州生物医学成像重点实验室）；Zhejiang Key Laboratory of Soft Matter Biomedical Materials（浙江软物质生物医学材料重点实验室）；Wenzhou Institute, University of Chinese Academy of Sciences（中国科学院大学温州研究院）；Postgraduate Training Base Alliance of Wenzhou Medical University（温州医学院研究生联合培养基地联盟）
Automating Parameter Selection in Deep Image Prior for Fluorescence Microscopy Image Denoising via Similarity-Based Parameter Transfer
Institute for Applied Medical Informatics, Institute of Computational Neuroscience, Center for Biomedical Artificial Intelligence (bAIome), University Medical Center Hamburg-Eppendorf, Germany（应用医学信息学研究所、计算神经科学研究所、生物医学人工智能中心（bAIome）、汉堡-埃彭多夫大学医学中心，德国）；Institute for Biomedical Imaging, Hamburg University of Technology, Hamburg, Germany（生物医学成像研究所、汉堡技术大学、汉堡，德国）；Department of Biomedical Imaging, University Medical Center Hamburg-Eppendorf, Germany（生物医学成像系、汉堡-埃彭多夫大学医学中心，德国）；Fraunhofer Research Institution for Individualized Medical Technology（弗劳恩霍夫个性化医疗技术研究机构）；Institute of Medical Microbiology, Virology and Hygiene, University Medical Center Hamburg-Eppendorf, Germany（医学微生物学研究所、病毒学与卫生学、汉堡-埃彭多夫大学医学中心，德国）；Institute of Biochemistry and Molecular Cell Biology, University Medical Center Hamburg-Eppendorf, Germany（生物化学与分子细胞生物学研究所、汉堡-埃彭多夫大学医学中心，德国）
In fluorescence microscopy live-cell imaging, there is a critical trade-off between the signal-to-noise ratio and spatial resolution on one side, and the integrity of the biological sample on the other side. To obtain clean high-resolution (HR) images, one can either use microscopy techniques, such as structured-illumination microscopy (SIM), or apply denoising and super-resolution (SR) algorithms. However, the former option requires multiple shots that can damage the samples, and although efficient deep learning based algorithms exist for the latter option, no benchmark exists to evaluate the…
Deep Joint Denoising and Detection for Enhanced Intracellular Particle Analysis
Erasmus University Medical Center（伊拉斯姆斯大学医学中心）；Utrecht University（乌得勒支大学）；University of New South Wales (UNSW)（新南威尔士大学）；NVIDIA Corporation（英伟达公司）；SURFsara
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California NanoSystems Institute（加州纳米系统研究所）；David Geffen School of Medicine（戴维·格芬医学院）
Histological analysis of tissue samples is one of the most widely used methods for disease diagnosis. After taking a sample from a patient, it goes through a lengthy and laborious preparation, which stains the tissue to visualize different histological features under a microscope. Here, we demonstrate a label-free approach to create a virtually-stained microscopic image using a single wide-field auto-fluorescence image of an unlabeled tissue sample, bypassing the standard histochemical staining process, saving time and cost. This method is based on deep learning, and uses a convolutional neura…
Confocal microscopy is essential for histopathologic cell visualization and quantification. Despite its significant role in biology, fluorescence confocal microscopy suffers from the presence of inherent noise during image acquisition. Non-local patch-wise Bayesian mean filtering (NLB) was until recently the state-of-the-art denoising approach. However, classic denoising methods have been outperformed by neural networks in recent years. In this work, we propose to exploit the strengths of NLB in the framework of Bayesian deep learning. We do so by designing a convolutional neural network and t…
CyclePose -- Leveraging Cycle-Consistency for Annotation-Free Nuclei Segmentation in Fluorescence Microscopy
Friedrich-Alexander-Universität (FAU)（弗里德里希-亚历山大-埃尔朗根-纽伦堡大学（FAU））；Universität Würzburg（维尔茨堡大学）；Center for AI and Data Science (CAIDAS)（人工智能与数据科学中心（CAIDAS））
University of California San Diego（加州大学圣迭戈分校）；National Taiwan University（台湾大学）；University of Macau（澳门大学）；National Taiwan University Hospital and College of Medicine（台湾大学医学院附设医院暨医学院）
As the rapid growth of high-speed and deep-tissue imaging in biomedical research, it is urgent to find a robust and effective denoising method to retain morphological features for further texture analysis and segmentation. Conventional denoising filters and models can easily suppress perturbative noises in high contrast images. However, for low photon budget multi-photon images, high detector gain will not only boost signals, but also bring huge background noises. In such stochastic resonance regime of imaging, sub-threshold signals may be detectable with the help of noises. Therefore, a denoi…
MDiff-FMT: Morphology-aware Diffusion Model for Fluorescence Molecular Tomography with Small-scale Datasets
School of Computer and Information Technology, Shanxi University（山西大学计算机与信息技术学院）；School of Biological and Medical Engineering, Beihang University（北京航空航天大学生物与医学工程学院）
University of Edinburgh（爱丁堡大学）；Heriot-Watt University（赫瑞-瓦特大学）
Autofluorescence lifetime images reveal unique characteristics of endogenous fluorescence in biological samples. Comprehensive understanding and clinical diagnosis rely on co-registration with the gold standard, histology images, which is extremely challenging due to the difference of both images. Here, we show an unsupervised image-to-image translation network that significantly improves the success of the co-registration using a conventional optimisation-based regression network, applicable to autofluorescence lifetime images at different emission wavelengths. A preliminary blind comparison …
We leverage deep learning techniques to jointly denoise and super-resolve biomedical images acquired with fluorescence microscopy. We develop a deep learning algorithm based on the networks and method described in the recent W2S paper to solve a joint denoising and super-resolution problem. Specifically, we address the restoration of SIM images from widefield images. Our TensorFlow model is trained on the W2S dataset of cell images and is made accessible online in this repository: https://github.com/mchatton/w2s-tensorflow. On test images, the model shows a visually-convincing denoising and in…
Patch-Based Encoder-Decoder Architecture for Automatic Transmitted Light to Fluorescence Imaging Transition: Contribution to the LightMyCells Challenge
AGH University of Krakow（克拉科夫AGH科技大学）；University of Applied Sciences Western Switzerland（瑞士西应用科学大学）；University of Geneva（日内瓦大学）
Volumetric imaging of samples using fluorescence microscopy plays an important role in various fields including physical, medical and life sciences. Here we report a deep learning-based volumetric image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope at arbitrary axial positions within the sample volume. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume over …
Volume Tells: Dual Cycle-Consistent Diffusion for 3D Fluorescence Microscopy De-noising and Super-Resolution
Department of Electrical Engineering, City University of Hong Kong（香港城市大学电气工程系）；Centre for Intelligent Multidimensional Data Analysis, Hong Kong Science Park（香港科学园智能多维数据分析中心）；Department of Biology, Hong Kong Baptist University（香港浸会大学生物学系）
Indian Institute of Technology Guwahati（印度理工学院古瓦哈蒂分校）；UiT The Arctic University of Norway（挪威特罗姆瑟大学（UiT 挪威北极大学））；Indian Institute of Technology Dhanbad（印度理工学院丹巴德分校）
Scanning acoustic microscopy (SAM) has been employed since microscopic images are widely used for biomedical or materials research. Acoustic imaging is an important and well-established method used in nondestructive testing (NDT), bio-medical imaging, and structural health monitoring.The imaging is frequently carried out with signals of low amplitude, which might result in leading that are noisy and lacking in details of image information. In this work, we attempted to analyze SAM images acquired from low amplitude signals and employed a block matching filter over time domain signals to obtain…
Prediction of soil fertility parameters using USB-microscope imagery and portable X-ray fluorescence spectrometry
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）；Bidhan Chandra Krishi Viswavidyalaya（比丹·钱德拉农业大学）；Washington State University（华盛顿州立大学）；Space Applications Centre, Indian Space Research Organisation（印度空间研究组织空间应用中心）；University of Guelph（圭尔夫大学）；Georgia Southern University（佐治亚南方大学）；Louisiana State University（路易斯安那州立大学）；Federal University of Lavras（拉夫拉斯联邦大学）
Denoising Diffusion Probabilistic Models for Image Inpainting of Cell Distributions in the Human Brain
Institute of Neuroscience and Medicine (INM-1), Research Centre Jülich（于利希研究中心神经科学与医学研究所（INM-1））；Helmholtz AI, Research Centre Jülich（于利希研究中心亥姆霍兹人工智能）；Institute of Computer Science, Heinrich-Heine-University Düsseldorf（杜塞尔多夫海因里希·海涅大学计算机科学研究所）
Northeastern University（东北大学）；Stevens Institute of Technology（史蒂文斯理工学院）；Huazhong University of Science and Technology（华中科技大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Chengdu University of Information Technology（成都信息工程大学）
Environmental Microorganism Data Set Fifth Version (EMDS-5) is a microscopic image dataset including original Environmental Microorganism (EM) images and two sets of Ground Truth (GT) images. The GT image sets include a single-object GT image set and a multi-object GT image set. The EMDS-5 dataset has 21 types of EMs, each of which contains 20 original EM images, 20 single-object GT images and 20 multi-object GT images. EMDS-5 can realize to evaluate image preprocessing, image segmentation, feature extraction, image classification and image retrieval functions. In order to prove the effectiven…
Technical University of Munich（慕尼黑工业大学）；Georg-August-University Goettingen（哥廷根大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Institute of AI for Health, Helmholtz Munich - German Research Center for Environmental Health（亥姆霍兹慕尼黑中心-德国环境健康研究中心健康人工智能研究所）；Helmholtz AI, Helmholtz Munich - German Research Center for Environmental Health（亥姆霍兹慕尼黑中心-德国环境健康研究中心亥姆霍兹人工智能）
Light-sheet fluorescence microscopy (LSFM) is a cutting-edge volumetric imaging technique that allows for three-dimensional imaging of mesoscopic samples with decoupled illumination and detection paths. Although the selective excitation scheme of such a microscope provides intrinsic optical sectioning that minimizes out-of-focus fluorescence background and sample photodamage, it is prone to light absorption and scattering effects, which results in uneven illumination and striping artifacts in the images adversely. To tackle this issue, in this paper, we propose a blind stripe artifact removal …
We demonstrate a deep learning-based offline autofocusing method, termed Deep-R, that is trained to rapidly and blindly autofocus a single-shot microscopy image of a specimen that is acquired at an arbitrary out-of-focus plane. We illustrate the efficacy of Deep-R using various tissue sections that were imaged using fluorescence and brightfield microscopy modalities and demonstrate snapshot autofocusing under different scenarios, such as a uniform axial defocus as well as a sample tilt within the field-of-view. Our results reveal that Deep-R is significantly faster when compared with standard …
A Fractional Variational Approach to Spectral Filtering Using the Fourier Transform
Laboratory of Mathematical-Chemistry, Institute of Chemistry, Universidade Federal de Alfenas (UNIFAL)（数学化学实验室，化学学院，联邦阿尔费纳斯大学）；Institute of Exact Science, Universidade Federal de Alfenas (UNIFAL)（精确科学学院，联邦阿尔费纳斯大学）
Beijing Institute of Technology（北京理工大学）；Duke University（杜克大学）
Benefiting from its single-photon sensitivity, single-photon avalanche diode (SPAD) array has been widely applied in various fields such as fluorescence lifetime imaging and quantum computing. However, large-scale high-fidelity single-photon imaging remains a big challenge, due to the complex hardware manufacture craft and heavy noise disturbance of SPAD arrays. In this work, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging over an order of magnitude, with significant enhancement of bit depth and imaging quality. We first studied the complex photon flow mod…
Single color digital H&E staining with In-and-Out Net
University of Texas at Austin（德克萨斯大学奥斯汀分校）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；The University of Texas MD Anderson Cancer Center（德克萨斯大学 MD 安德森癌症中心）
