Multiple stains are usually used to highlight biological substances in biomedical image analysis. To decompose multiple stains for co-localization quantification, blind source separation is usually performed. Prior model-based stain separation methods usually rely on stains' spatial distributions over an image and may fail to solve the co-localization problem. With the advantage of machine learning, deep generative models are used for this purpose. Since prior knowledge of imaging models is ignored in purely data-driven solutions, these methods may be sub-optimal. In this study, a novel learning-based blind source separation framework is proposed, where the physical model of biomedical imaging is incorporated to regularize the learning process. The introduced model-relevant adversarial loss couples all generators in the framework and limits the capacities of the generative models. Further more, a training algorithm is innovated for the proposed framework to avoid inter-generator confusion during learning. This paper particularly takes fluorescence unmixing in fluorescence microscopy images as an application example of the proposed framework. Qualitative and quantitative experimentation on a public fluorescence microscopy image set demonstrates the superiority of the proposed method over both prior model-based approaches and learning-based methods.
ETH Zurich（苏黎世联邦理工学院）；University Hospital（大学医院）；University of Zurich（苏黎世大学）；Uppsala University（乌普萨拉大学）
Fluorescence microscopy images contain several channels, each indicating a marker staining the sample. Since many different marker combinations are utilized in practice, it has been challenging to apply deep learning based segmentation models, which expect a predefined channel combination for all training samples as well as at inference for future application. Recent work circumvents this problem using a modality attention approach to be effective across any possible marker combination. However, for combinations that do not exist in a labeled training dataset, one cannot have any estimation of…
FluoCLIP: Stain-Aware Focus Quality Assessment in Fluorescence Microscopy
Division of AI and Software, Ewha Womans University（梨花女子大学人工智能与软件学部）；Bioimaging Data Curation Center (BDCC), Department of Life Science, Ewha Womans University（梨花女子大学生命科学系生物成像数据管理中心）；Analytical Solution Team, Daesang Corporation（大象集团分析解决方案团队）
Tencent AI Lab（腾讯AI实验室）；University of Connecticut（康涅狄格大学）；The Fourth Hospital of Hebei Medical University（河北医科大学第四医院）；West China Hospital, Sichuan University（四川大学华西医院）
Artificial Intelligence (AI)-powered pathology is a revolutionary step in the world of digital pathology and shows great promise to increase both diagnosis accuracy and efficiency. However, defocus and motion blur can obscure tissue or cell characteristics hence compromising AI algorithms'accuracy and robustness in analyzing the images. In this paper, we demonstrate a deep-learning-based approach that can alleviate the defocus and motion blur of a microscopic image and output a sharper and cleaner image with retrieved fine details without prior knowledge of the blur type, blur extent and patho…
Imaging assays of cellular function, especially those using fluorescent stains, are ubiquitous in the biological and medical sciences. Despite advances in computer vision, such images are often analyzed using only manual or rudimentary automated processes. Watershed-based segmentation is an effective technique for identifying objects in images; it outperforms commonly used image analysis methods, but requires familiarity with computer-vision techniques to be applied successfully. In this report, we present and implement a watershed-based image analysis and classification algorithm in a GUI, en…
Fourier ptychographic microscopy (FPM) is a computational approach geared towards creating high-resolution and large field-of-view images without mechanical scanning. To acquire color images of histology slides, it often requires sequential acquisitions with red, green, and blue illuminations. The color reconstructions often suffer from coherent artifacts that are not presented in regular incoherent microscopy images. As a result, it remains a challenge to employ FPM for digital pathology applications, where resolution and color accuracy are of critical importance. Here we report a deep learni…
Visualizing the details of different cellular structures is of great importance to elucidate cellular functions. However, it is challenging to obtain high quality images of different structures directly due to complex cellular environments. Fluorescence staining is a popular technique to label different structures but has several drawbacks. In particular, label staining is time consuming and may affect cell morphology, and simultaneous labels are inherently limited. This raises the need of building computational models to learn relationships between unlabeled microscopy images and labeled fluo…
Confocal microscopy is essential for histopathologic cell visualization and quantification. Despite its significant role in biology, fluorescence confocal microscopy suffers from the presence of inherent noise during image acquisition. Non-local patch-wise Bayesian mean filtering (NLB) was until recently the state-of-the-art denoising approach. However, classic denoising methods have been outperformed by neural networks in recent years. In this work, we propose to exploit the strengths of NLB in the framework of Bayesian deep learning. We do so by designing a convolutional neural network and t…
We demonstrate a deep learning-based offline autofocusing method, termed Deep-R, that is trained to rapidly and blindly autofocus a single-shot microscopy image of a specimen that is acquired at an arbitrary out-of-focus plane. We illustrate the efficacy of Deep-R using various tissue sections that were imaged using fluorescence and brightfield microscopy modalities and demonstrate snapshot autofocusing under different scenarios, such as a uniform axial defocus as well as a sample tilt within the field-of-view. Our results reveal that Deep-R is significantly faster when compared with standard …
Fluorescence microscopy plays a vital role in understanding the subcellular structures of living cells. However, it requires considerable effort in sample preparation related to chemical fixation, staining, cost, and time. To reduce those factors, we present a virtual fluorescence staining method based on deep neural networks (VirFluoNet) to transform fluorescence images of molecular labels into other molecular fluorescence labels in the same field-of-view. To achieve this goal, we develop and train a conditional generative adversarial network (cGAN) to perform digital fluorescence imaging dem…
Segmentation-free integration of nuclei morphology and spatial transcriptomics for retinal images
Dept. of Information Technology（信息技术系）；Dept. of Molecular, Cell and Developmental Biology（分子、细胞与发育生物学系）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
University of Edinburgh（爱丁堡大学）；Heriot-Watt University（赫瑞-瓦特大学）
Autofluorescence lifetime images reveal unique characteristics of endogenous fluorescence in biological samples. Comprehensive understanding and clinical diagnosis rely on co-registration with the gold standard, histology images, which is extremely challenging due to the difference of both images. Here, we show an unsupervised image-to-image translation network that significantly improves the success of the co-registration using a conventional optimisation-based regression network, applicable to autofluorescence lifetime images at different emission wavelengths. A preliminary blind comparison …
MTCurv: Deep learning for direct microtubule curvature mapping in noisy fluorescence microscopy images
TIAD Laboratory, Sciences and Technology Faculty, Sultan Moulay Slimane Univ.（TIAD实验室，科技与技术学院，苏丹穆莱·斯利曼大学）；CNRS, Univ. Rennes, IGDR (Institut de Génétique et Développement de Rennes), UMR6290（法国国家科学研究中心，雷恩大学，雷恩基因与发育研究所，UMR6290）
Arizona State University（亚利桑那州立大学）；Barrow Neurological Institute（巴罗神经学研究所）；St. Joseph’s Hospital and Medical Center（圣约瑟夫医院和医疗中心）
Confocal laser endomicroscopy (CLE) allow on-the-fly in vivo intraoperative imaging in a discreet field of view, especially for brain tumors, rather than extracting tissue for examination ex vivo with conventional light microscopy. Fluorescein sodium-driven CLE imaging is more interactive, rapid, and portable than conventional hematoxylin and eosin (H&E)-staining. However, it has several limitations: CLE images may be contaminated with artifacts (motion, red blood cells, noise), and neuropathologists are mainly trained on colorful stained histology slides like H&E while the CLE images are gray…
Virtual birefringence imaging and histological staining of amyloid deposits in label-free tissue using autofluorescence microscopy and deep learning
University of California, Los Angeles（加州大学洛杉矶分校）；California NanoSystems Institute (CNSI), University of California, Los Angeles（加州大学洛杉矶分校加州纳米系统研究所）；Fatih Sultan Mehmet Vakif University（法提赫·苏丹·穆罕默德·瓦克夫大学）；David Geffen School of Medicine at the University of California, Los Angeles（加州大学洛杉矶分校大卫·格芬医学院）；Hadassah Hebrew University Medical Center（哈达萨希伯来大学医学中心）；Keck School of Medicine, University of Southern California（南加州大学凯克医学院）
ETH Zurich（苏黎世联邦理工学院）；University Hospital and University of Zurich（苏黎世大学医院和苏黎世大学）；Uppsala University（乌普萨拉大学）
Fluorescence microscopy allows for a detailed inspection of cells, cellular networks, and anatomical landmarks by staining with a variety of carefully-selected markers visualized as color channels. Quantitative characterization of structures in acquired images often relies on automatic image analysis methods. Despite the success of deep learning methods in other vision applications, their potential for fluorescence image analysis remains underexploited. One reason lies in the considerable workload required to train accurate models, which are normally specific for a given combination of markers…
Fluorescence microscopy is a key driver to promote discoveries of biomedical research. However, with the limitation of microscope hardware and characteristics of the observed samples, the fluorescence microscopy images are susceptible to noise. Recently, a few self-supervised deep learning (DL) denoising methods have been proposed. However, the training efficiency and denoising performance of existing methods are relatively low in real scene noise removal. To address this issue, this paper proposed self-supervised image denoising method Noise2SR (N2SR) to train a simple and effective image den…
Autonomous labeling of surgical resection margins using a foundation model
University of California, Los Angeles（加州大学洛杉矶分校）；Fatih Sultan Mehmet Vakif University（法提赫·苏丹·穆罕默德·瓦克夫大学）；University of California, Davis（加州大学戴维斯分校）；Hadassah Hebrew University Medical Center（哈达萨希伯来大学医学中心）
We report a novel lensless on-chip microscopy platform based on near-field blind ptychographic modulation. In this platform, we place a thin diffuser in between the object and the image sensor for light wave modulation. By blindly scanning the unknown diffuser to different x-y positions, we acquire a sequence of modulated intensity images for quantitative object recovery. Different from previous ptychographic implementations, we employ a unit magnification configuration with a Fresnel number of ~50,000, which is orders of magnitude higher than previous ptychographic setups. The unit magnificat…
Technical University of Munich（慕尼黑工业大学）；Georg-August-University Goettingen（哥廷根大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Institute of AI for Health, Helmholtz Munich - German Research Center for Environmental Health（亥姆霍兹慕尼黑中心-德国环境健康研究中心健康人工智能研究所）；Helmholtz AI, Helmholtz Munich - German Research Center for Environmental Health（亥姆霍兹慕尼黑中心-德国环境健康研究中心亥姆霍兹人工智能）
Light-sheet fluorescence microscopy (LSFM) is a cutting-edge volumetric imaging technique that allows for three-dimensional imaging of mesoscopic samples with decoupled illumination and detection paths. Although the selective excitation scheme of such a microscope provides intrinsic optical sectioning that minimizes out-of-focus fluorescence background and sample photodamage, it is prone to light absorption and scattering effects, which results in uneven illumination and striping artifacts in the images adversely. To tackle this issue, in this paper, we propose a blind stripe artifact removal …
Analysis of microscope images is a tedious work which requires patience and time, usually done manually by the microscopist after data collection. Here we introduce an approach of automatic image analysis, which is based on locally applied Fourier Transform and Machine Learning methods. In this approach, a whole image is scanned by a local moving window with defined size and the 2D Fourier Transform is calculated for each window. Then, all the Local Fourier Transforms are fed into Machine Learning processing. Firstly, a number of components in the data is estimated from Principal Component Ana…
