Recently developed methods for rapid continuous volumetric two-photon microscopy facilitate the observation of neuronal activity in hundreds of individual neurons and changes in blood flow in adjacent blood vessels across a large volume of living brain at unprecedented spatio-temporal resolution. However, the high imaging rate necessitates fully automated image analysis, whereas tissue turbidity and photo-toxicity limitations lead to extremely sparse and noisy imagery. In this work, we extend a recently proposed deep learning volumetric blood vessel segmentation network, such that it supports temporal analysis. With this technology, we are able to track changes in cerebral blood volume over time and identify spontaneous arterial dilations that propagate towards the pial surface. This new capability is a promising step towards characterizing the hemodynamic response function upon which functional magnetic resonance imaging (fMRI) is based.
Tel Aviv University（特拉维夫大学）；Facebook AI Research（脸书人工智能研究院）
The task of blood vessel segmentation in microscopy images is crucial for many diagnostic and research applications. However, vessels can look vastly different, depending on the transient imaging conditions, and collecting data for supervised training is laborious. We present a novel deep learning method for unsupervised segmentation of blood vessels. The method is inspired by the field of active contours and we introduce a new loss term, which is based on the morphological Active Contours Without Edges (ACWE) optimization method. The role of the morphological operators is played by novel pool…
University of Nevada, Reno（内华达大学里诺分校）；Houston Eye Associates（休斯顿眼科协会）
High fidelity segmentation of both macro and microvascular structure of the retina plays a pivotal role in determining degenerative retinal diseases, yet it is a difficult problem. Due to successive resolution loss in the encoding phase combined with the inability to recover this lost information in the decoding phase, autoencoding based segmentation approaches are limited in their ability to extract retinal microvascular structure. We propose RV-GAN, a new multi-scale generative architecture for accurate retinal vessel segmentation to alleviate this. The proposed architecture uses two generat…
Capillaries are the smallest vessels in the body responsible for delivering oxygen and nutrients to surrounding cells. Various life-threatening diseases are known to alter the density of healthy capillaries and the flow velocity of erythrocytes within the capillaries. In previous studies, capillary density and flow velocity were manually assessed by trained specialists. However, manual analysis of a standard 20-second microvascular video requires 20 minutes on average and necessitates extensive training. Thus, manual analysis has been reported to hinder the application of microvascular microsc…
University of Houston（休斯顿大学）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Maps of brain microarchitecture are important for understanding neurological function and behavior, including alterations caused by chronic conditions such as neurodegenerative disease. Techniques such as knife-edge scanning microscopy (KESM) provide the potential for whole organ imaging at sub-cellular resolution. However, multi-terabyte data sizes make manual annotation impractical and automatic segmentation challenging. Densely packed cells combined with interconnected microvascular networks are a challenge for current segmentation algorithms. The massive size of high-throughput microscopy …
Purdue University（普渡大学）；Indiana University-Purdue University Indianapolis（印第安纳大学-普渡大学印第安纳波利斯分校）；Indiana University（印第安纳大学）
Fluorescence microscopy is an essential tool for the analysis of 3D subcellular structures in tissue. An important step in the characterization of tissue involves nuclei segmentation. In this paper, a two-stage method for segmentation of nuclei using convolutional neural networks (CNNs) is described. In particular, since creating labeled volumes manually for training purposes is not practical due to the size and complexity of the 3D data sets, the paper describes a method for generating synthetic microscopy volumes based on a spatially constrained cycle-consistent adversarial network. The prop…
Universidad Peruana Cayetano Heredia（秘鲁卡耶塔诺·埃雷迪亚大学）；University of Padova（帕多瓦大学）
Tuberculosis (TB), caused by a germ called Mycobacterium tuberculosis, is one of the most serious public health problems in Peru and the world. The development of this project seeks to facilitate and automate the diagnosis of tuberculosis by the MODS method and using lens-free microscopy, due they are easier to calibrate and easier to use (by untrained personnel) in comparison with lens microscopy. Thus, we employ a U-Net network in our collected dataset to perform the automatic segmentation of the TB cords in order to predict tuberculosis. Our initial results show promising evidence for autom…
University of California Berkeley（加州大学伯克利分校）；Lawrence Berkeley National Laboratory（劳伦斯伯克利国家实验室）
In the field of transmission electron microscopy, data interpretation often lags behind acquisition methods, as image processing methods often have to be manually tailored to individual datasets. Machine learning offers a promising approach for fast, accurate analysis of electron microscopy data. Here, we demonstrate a flexible two step pipeline for analysis of high resolution transmission electron microscopy data, which uses a U-Net for segmentation followed by a random forest for detection of stacking faults. Our trained U-Net is able to segment nanoparticle regions from amorphous background…
Going Beyond U-Net: Assessing Vision Transformers for Semantic Segmentation in Microscopy Image Analysis
Institute of Computer Science, University of Tartu（塔尔图大学计算机科学研究所）；Center for Artificial Intelligence, Technical University of Applied Sciences Würzburg-Schweinfurt（维尔茨堡-施韦因富特应用技术大学人工智能中心）；STACC
We propose a 3D convolutional neural network to simultaneously segment and detect cell nuclei in confocal microscopy images. Mirroring the co-dependency of these tasks, our proposed model consists of two serial components: the first part computes a segmentation of cell bodies, while the second module identifies the centers of these cells. Our model is trained end-to-end from scratch on a mouse parotid salivary gland stem cell nuclei dataset comprising 107 image stacks from three independent cell preparations, each containing several hundred individual cell nuclei in 3D. In our experiments, we …
Non-invasive detection of microvascular alterations in deep tissues in vivo provides critical information for clinical diagnosis and evaluation of a broad-spectrum of pathologies. Recently, the emergence of super-resolution ultrasound localization microscopy (ULM) offers new possibilities for clinical imaging of microvasculature at capillary level. Currently, the clinical utility of ULM on clinical ultrasound scanners is hindered by the technical limitations, such as long data acquisition time, and compromised tracking performance associated with low imaging frame-rate. Here we present an in-h…
Max Planck Institute of Molecular Cell Biology and Genetics（马克斯·普朗克分子细胞生物学与遗传学研究所）；Center for Systems Biology Dresden (CSBD)（德累斯顿系统生物学中心）；IT4Innovations, VŠB - Technical University of Ostrava（IT4Innovations，俄斯特拉发技术大学）；Fondatione Human Technopole（人类技术基金会）
Automatic detection and segmentation of objects in 2D and 3D microscopy data is important for countless biomedical applications. In the natural image domain, spatial embedding-based instance segmentation methods are known to yield high-quality results, but their utility for segmenting microscopy data is currently little researched. Here we introduce EmbedSeg, an embedding-based instance segmentation method which outperforms existing state-of-the-art baselines on 2D as well as 3D microscopy datasets. Additionally, we show that EmbedSeg has a GPU memory footprint small enough to train even on la…
Fluorescence microscopy has become a widely used tool for studying various biological structures of in vivo tissue or cells. However, quantitative analysis of these biological structures remains a challenge due to their complexity which is exacerbated by distortions caused by lens aberrations and light scattering. Moreover, manual quantification of such image volumes is an intractable and error-prone process, making the need for automated image analysis methods crucial. This paper describes a segmentation method for tubular structures in fluorescence microscopy images using convolutional neura…
Advances in fluorescence microscopy enable acquisition of 3D image volumes with better image quality and deeper penetration into tissue. Segmentation is a required step to characterize and analyze biological structures in the images and recent 3D segmentation using deep learning has achieved promising results. One issue is that deep learning techniques require a large set of groundtruth data which is impractical to annotate manually for large 3D microscopy volumes. This paper describes a 3D deep learning nuclei segmentation method using synthetic 3D volumes for training. A set of synthetic vol…
Nanjing University of Science and Technology（南京理工大学）
Optical coherence tomography angiography (OCTA) is a novel imaging modality that has been widely utilized in ophthalmology and neuroscience studies to observe retinal vessels and microvascular systems. However, publicly available OCTA datasets remain scarce. In this paper, we introduce the largest and most comprehensive OCTA dataset dubbed OCTA-500, which contains OCTA imaging under two fields of view (FOVs) from 500 subjects. The dataset provides rich images and annotations including two modalities (OCT/OCTA volumes), six types of projections, four types of text labels (age / gender / eye / d…
Institute of Imaging and Computer Vision, RWTH Aachen University（亚琛工业大学成像与计算机视觉研究所）
Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotatio…
Université de Tunis El Manar（突尼斯埃尔马纳尔大学）；Faculté des Sciences de Tunis（突尼斯科学学院）
Cells segmentation shows rapid growth in biology. Indeed, using the classical segmentation methods only is not enough to segment this type of images. In this manuscript, we will present a new method of ribosomes segmentation. A pre-treatment phase will precedes the segmentation process and after that a post-processing will proceed.
XLAB d.o.o.（XLAB公司）；University of Ljubljana（卢布尔雅那大学）
A key component towards an improved and fast cancer diagnosis is the development of computer-assisted tools. In this article, we present the solution that won the SegPC-2021 competition for the segmentation of multiple myeloma plasma cells in microscopy images. The labels used in the competition dataset were generated semi-automatically and presented noise. To deal with it, a heavy image augmentation procedure was carried out and predictions from several models were combined using a custom ensemble strategy. State-of-the-art feature extractors and instance segmentation architectures were used,…
European Bioinformatics Institute（欧洲生物信息学研究所）；European Molecular Biology Laboratory（欧洲分子生物学实验室）
We present AURA-net, a convolutional neural network (CNN) for the segmentation of phase-contrast microscopy images. AURA-net uses transfer learning to accelerate training and Attention mechanisms to help the network focus on relevant image features. In this way, it can be trained efficiently with a very limited amount of annotations. Our network can thus be used to automate the segmentation of datasets that are generally considered too small for deep learning techniques. AURA-net also uses a loss inspired by active contours that is well-adapted to the specificity of phase-contrast images, furt…
University of South Bohemia in České Budějovice（捷克布杰约维采南波希米亚大学）；Helmholtz Centre Potsdam GFZ German Research Centre for Geosciences（亥姆霍兹波茨坦中心德国地球科学研究中心）
Living cell segmentation from bright-field light microscopy images is challenging due to the image complexity and temporal changes in the living cells. Recently developed deep learning (DL)-based methods became popular in medical and microscopy image segmentation tasks due to their success and promising outcomes. The main objective of this paper is to develop a deep learning, U-Net-based method to segment the living cells of the HeLa line in bright-field transmitted light microscopy. To find the most suitable architecture for our datasets, a residual attention U-Net was proposed and compared w…
Slovak University of Technology in Bratislava（布拉迪斯拉发斯洛伐克理工大学）；CNRS（法国国家科学研究中心）；Univ. Montpellier（蒙彼利埃大学）
The automated segmentation and tracking of macrophages during their migration are challenging tasks due to their dynamically changing shapes and motions. This paper proposes a new algorithm to achieve automatic cell tracking in time-lapse microscopy macrophage data. First, we design a segmentation method employing space-time filtering, local Otsu's thresholding, and the SUBSURF (subjective surface segmentation) method. Next, the partial trajectories for cells overlapping in the temporal direction are extracted in the segmented images. Finally, the extracted trajectories are linked by consideri…
ETH Zurich（苏黎世联邦理工学院）；University Hospital（大学医院）；University of Zurich（苏黎世大学）；Uppsala University（乌普萨拉大学）
Fluorescence microscopy images contain several channels, each indicating a marker staining the sample. Since many different marker combinations are utilized in practice, it has been challenging to apply deep learning based segmentation models, which expect a predefined channel combination for all training samples as well as at inference for future application. Recent work circumvents this problem using a modality attention approach to be effective across any possible marker combination. However, for combinations that do not exist in a labeled training dataset, one cannot have any estimation of…
Oak Ridge National Lab (ORNL)（橡树岭国家实验室）；St. Jude Children’s Hospital（圣裘德儿童医院）
Microscopy imaging techniques are instrumental for characterization and analysis of biological structures. As these techniques typically render 3D visualization of cells by stacking 2D projections, issues such as out-of-plane excitation and low resolution in the $z$-axis may pose challenges (even for human experts) to detect individual cells in 3D volumes as these non-overlapping cells may appear as overlapping. In this work, we introduce a comprehensive method for accurate 3D instance segmentation of cells in the brain tissue. The proposed method combines the 2D YOLO detection method with a m…
Czech Technical University in Prague（布拉格捷克理工大学）；General Teaching Hospital（综合教学医院）；Institute of Physics of the Czech Academy of Sciences（捷克科学院物理研究所）
Infertility is becoming an issue for an increasing number of couples. The most common solution, in vitro fertilization, requires embryologists to carefully examine light microscopy images of human oocytes to determine their developmental potential. We propose an automatic system to improve the speed, repeatability, and accuracy of this process. We first localize individual oocytes and identify their principal components using CNN (U-Net) segmentation. Next, we calculate several descriptors based on geometry and texture. The final step is an SVM classifier. Both the segmentation and classificat…
PSL University（巴黎文理研究大学）；Mines Paris（巴黎矿业学院）；Centre for Computational Biology（计算生物学中心）；Institut Curie（居里研究所）；INSERM（法国国家健康与医学研究院）；University of Montpellier（蒙彼利埃大学）；IGH（人类遗传学研究所）；CNRS（法国国家科学研究中心）
Deep learning provides us with powerful methods to perform nucleus or cell segmentation with unprecedented quality. However, these methods usually require large training sets of manually annotated images, which are tedious and expensive to generate. In this paper we propose to use In Silico Labeling (ISL) as a pretraining scheme for segmentation tasks. The strategy is to acquire label-free microscopy images (such as bright-field or phase contrast) along fluorescently labeled images (such as DAPI or CellMask). We then train a model to predict the fluorescently labeled images from the label-free…
Universidad Peruana Cayetano Heredia（秘鲁卡耶塔诺埃雷迪亚大学）；University of Padova（帕多瓦大学）
One of the most serious public health problems in Peru and worldwide is Tuberculosis (TB), which is produced by a bacterium known as Mycobacterium tuberculosis. The purpose of this work is to facilitate and automate the diagnosis of tuberculosis using the MODS method and using lens-free microscopy, as it is easier to calibrate and easier to use by untrained personnel compared to lens microscopy. Therefore, we employed a U-Net network on our collected data set to perform automatic segmentation of cord shape bacterial accumulation and then predict tuberculosis. Our results show promising evidenc…
The coronary microvascular disease poses a great threat to human health. Computer-aided analysis/diagnosis systems help physicians intervene in the disease at early stages, where 3D vessel segmentation is a fundamental step. However, there is a lack of carefully annotated dataset to support algorithm development and evaluation. On the other hand, the commonly-used U-Net structures often yield disconnected and inaccurate segmentation results, especially for small vessel structures. In this paper, motivated by the data scarcity, we first construct two large-scale vessel segmentation datasets con…
Stable deep neural network architectures for mitochondria segmentation on electron microscopy volumes
University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU））；Donostia International Physics Center (DIPC)（多诺斯蒂亚国际物理中心（DIPC））；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；Instituto de Investigación Sanitaria Gregorio Marañón（格雷戈里奥·马拉尼翁健康研究所）；Ikerbasque, Basque Foundation for Science（巴斯克科学基金会（Ikerbasque））
European Laboratory for Non-Linear Spectroscopy (LENS), University of Florence（佛罗伦萨大学欧洲非线性光谱实验室）；Bioretics SrL（Bioretics SrL公司）；National Institute of Optics, National Research Council (INO-CNR)（国家研究委员会国家光学研究所）；Department of Physics（物理系）
Semantic segmentation of neuronal structures in 3D high-resolution fluorescence microscopy imaging of the human brain cortex can take advantage of bidimensional CNNs, which yield good results in neuron localization but lead to inaccurate surface reconstruction. 3D CNNs, on the other hand, would require manually annotated volumetric data on a large scale and hence considerable human effort. Semi-supervised alternative strategies which make use only of sparse annotations suffer from longer training times and achieved models tend to have increased capacity compared to 2D CNNs, needing more ground…
Accurate segmentation of 3-D cell nuclei in microscopy images is essential for the study of nuclear organization, gene expression, and cell morphodynamics. Current image segmentation methods are challenged by the complexity and variability of microscopy images and often over-segment or under-segment the cell nuclei. Thus, there is a need to improve segmentation accuracy and reliability, as well as the level of automation. In this paper, we propose a new automated algorithm for robust segmentation of 3-D cell nuclei using the concepts of random walk, graph theory, and mathematical morphology as…
Practical Guidelines for Cell Segmentation Models Under Optical Aberrations in Microscopy
The Fifth Affiliated Hospital of Wenzhou Medical University（温州医科大学附属第五医院）；Tsinghua University（清华大学）；Hangzhou Dianzi University（杭州电子科技大学）；Shandong University（山东大学）
Wayne State University（韦恩州立大学）；NanoBioScience Institute（纳米生物科学研究所）；College of Engineering（工学院）；School of Medicine（医学院）；Center for Molecular Medicine & Genetics（分子医学与遗传学中心）；University of Vermont College of Medicine（佛蒙特大学医学院）；Karolinska Institute（卡罗林斯卡学院）
Deep Neural Networks (DNN) have been widely used to carry out segmentation tasks in both electron and light microscopy. Most DNNs developed for this purpose are based on some variation of the encoder-decoder type U-Net architecture, in combination with residual blocks to increase ease of training and resilience to gradient degradation. Here we introduce Res-CR-Net, a type of DNN that features residual blocks with either a bundle of separable atrous convolutions with different dilation rates or a convolutional LSTM. The number of filters used in each residual block and the number of blocks are …
Sabanci University（萨班哲大学）；ARM Ltd.（ARM有限公司）；University of Zurich（苏黎世大学）；University of Utah（犹他大学）；Izmir University of Economics（伊兹密尔经济大学）；University of Rochester（罗切斯特大学）
Data driven segmentation is an important initial step of shape prior-based segmentation methods since it is assumed that the data term brings a curve to a plausible level so that shape and data terms can then work together to produce better segmentations. When purely data driven segmentation produces poor results, the final segmentation is generally affected adversely. One challenge faced by many existing data terms is due to the fact that they consider only pixel intensities to decide whether to assign a pixel to the foreground or to the background region. When the distributions of the foregr…
Leveraging Frequency Domain Learning in 3D Vessel Segmentation
Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；Zhongguancun Laboratory（中关村实验室）；School of Computing, National University of Singapore（新加坡国立大学计算学院）；Department of Computer Science, University of Hong Kong（香港大学计算机科学系）；Deepwise AI Lab（深睿医疗人工智能实验室）；HwaMei Hospital, University of Chinese Academy of Sciences（中国科学院大学华美医院）；Ningbo Institute of Life and Health Industry, UCAS（中国科学院大学宁波生命健康产业研究院）；School of Mathematical Sciences, Beihang University（北京航空航天大学数学科学学院）
Delft University of Technology（代尔夫特理工大学）；Rotterdam Eye Hospital（鹿特丹眼科医院）；Amphia Hospital（安菲娅医院）；Erasmus Medical Center Rotterdam（鹿特丹伊拉斯谟医学中心）；Novo Research Consultancy（诺沃研究咨询公司）
To estimate the corneal endothelial parameters from specular microscopy images depicting cornea guttata (Fuchs dystrophy), we propose a new deep learning methodology that includes a novel attention mechanism named feedback non-local attention (fNLA). Our approach first infers the cell edges, then selects the cells that are well detected, and finally applies a postprocessing method to correct mistakes and provide the binary segmentation from which the corneal parameters are estimated (cell density [ECD], coefficient of variation [CV], and hexagonality [HEX]). In this study, we analyzed 1203 ima…
University of Pennsylvania（宾夕法尼亚大学）；Brookhaven National Laboratory（布鲁克海文国家实验室）；University of Puerto Rico（波多黎各大学）
Cutting edge deep learning techniques allow for image segmentation with great speed and accuracy. However, application to problems in materials science is often difficult since these complex models may have difficultly learning physical parameters. In situ electron microscopy provides a clear platform for utilizing automated image analysis. In this work we consider the case of studying coarsening dynamics in supported nanoparticles, which is important for understanding e.g. the degradation of industrial catalysts. By systematically studying dataset preparation, neural network architecture, and…
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；Stein Eye Institute（斯坦眼科研究所）；VoxelCloud, Inc.（体素云公司）
The retinal vasculature provides important clues in the diagnosis and monitoring of systemic diseases including hypertension and diabetes. The microvascular system is of primary involvement in such conditions, and the retina is the only anatomical site where the microvasculature can be directly observed. The objective assessment of retinal vessels has long been considered a surrogate biomarker for systemic vascular diseases, and with recent advancements in retinal imaging and computer vision technologies, this topic has become the subject of renewed attention. In this paper, we present a novel…
gACSON software for automated segmentation and morphology analyses of myelinated axons in 3D electron microscopy
Department of Information Technology, Uppsala University（信息科技系，乌普萨拉大学）；Biomedical Imaging Unit, A.I.Virtanen Institute for Molecular Sciences, University of Eastern Finland（生物医学成像单元，A.I.维塔纳分子科学研究所，东芬兰大学）；Electron Microscopy Unit, Institute of Biotechnology, University of Helsinki（电子显微镜单元，生物技术研究所，赫尔辛基大学）
SELMA3D challenge: Self-supervised learning for 3D light-sheet microscopy image segmentation
Institute for Tissue Engineering and Regenerative Medicine, Helmholtz Center Munich, German Research Center for Environmental Health（亥姆霍兹慕尼黑中心环境健康研究所组织工程与再生医学研究所）；TUM School of Computation, Information and Technology (CIT), Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；University of Zurich（苏黎世大学）；Shanghai University of Finance and Economics（上海财经大学）；BGI Research（华大基因研究院）；Zhejiang University（浙江大学）；Sunnybrook Research Institute, University of Toronto（多伦多大学桑尼布鲁克研究所）
Center for Systems Biology Dresden (CSBD)（德累斯顿系统生物学中心）；Max-Planck Institute of Molecular Cell Biology and Genetics（马克斯·普朗克分子细胞生物学与遗传学研究所）
Deep learning (DL) has arguably emerged as the method of choice for the detection and segmentation of biological structures in microscopy images. However, DL typically needs copious amounts of annotated training data that is for biomedical projects typically not available and excessively expensive to generate. Additionally, tasks become harder in the presence of noise, requiring even more high-quality training data. Hence, we propose to use denoising networks to improve the performance of other DL-based image segmentation methods. More specifically, we present ideas on how state-of-the-art sel…
Deep-learning algorithms enable precise image recognition based on high-dimensional hierarchical image features. Here, we report the development and implementation of a deep-learning-based image segmentation algorithm in an autonomous robotic system to search for two-dimensional (2D) materials. We trained the neural network based on Mask-RCNN on annotated optical microscope images of 2D materials (graphene, hBN, MoS2, and WTe2). The inference algorithm is run on a 1024 x 1024 px2 optical microscope images for 200 ms, enabling the real-time detection of 2D materials. The detection process is ro…
Acoustic-resolution photoacoustic microscopy (AR-PAM) is a promising tool for microvascular imaging. In the focal region, resolution of AR-PAM is determined by the ultrasound transducer and ultimately limited by acoustic diffraction. In the out-of-focus region, resolution deteriorates with increasing distance from the focal plane, which restricts depth of focus (DOF). Besides, a trade-off exists between resolution and DOF. Previously, synthetic aperture focusing technique (SAFT) and/or deconvolution methods have been demonstrated to enhance AR-PAM images. However, they suffer from issues in lo…
In this paper, we present a novel region-based segmentation method for cortical spreading depressions in 2-photon microscopy images. Fluorescent microscopy has become an important tool in neuroscience, but segmentation approaches are challenged by the opaque properties and structures of brain tissue. These challenges are made more extreme when segmenting events such as cortical spreading depressions, where low signal-to-noise ratios and intensity inhomogeneity dominate images. The method we propose uses a local intensity similarity measure that takes advantage of normalized Euclidean and geode…
