The automated segmentation and tracking of macrophages during their migration are challenging tasks due to their dynamically changing shapes and motions. This paper proposes a new algorithm to achieve automatic cell tracking in time-lapse microscopy macrophage data. First, we design a segmentation method employing space-time filtering, local Otsu's thresholding, and the SUBSURF (subjective surface segmentation) method. Next, the partial trajectories for cells overlapping in the temporal direction are extracted in the segmented images. Finally, the extracted trajectories are linked by considering their direction of movement. The segmented images and the obtained trajectories from the proposed method are compared with those of the semi-automatic segmentation and manual tracking. The proposed tracking achieved 97.4% of accuracy for macrophage data under challenging situations, feeble fluorescent intensity, irregular shapes, and motion of macrophages. We expect that the automatically extracted trajectories of macrophages can provide pieces of evidence of how macrophages migrate depending on their polarization modes in the situation, such as during wound healing.
Faculdade de Ciências da Universidade do Porto（波尔图大学理学院）
Analysis of microscopy images is one important tool in many fields of biomedical research, as it allows the quantification of a multitude of parameters at the cellular level. However, manual counting of these images is both tiring and unreliable and ultimately very time-consuming for biomedical researchers. Not only does this slow down the overall research process, it also introduces counting errors due to a lack of objectivity and consistency inherent to the researchers own human nature. This thesis addresses this issue by automatically determining infection indexes of macrophages parasite by…
University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Sydney Nano Institute（悉尼纳米研究所）
Cell segmentation and tracking in microscopy images are of great significance to new discoveries in biology and medicine. In this study, we propose a novel approach to combine cell segmentation and cell tracking into a unified end-to-end deep learning based framework, where cell detection and segmentation are performed with a current instance segmentation pipeline and cell tracking is implemented by integrating Siamese Network with the pipeline. Besides, tracking performance is improved by incorporating spatial information into the network and fusing spatial and visual prediction. Our approach…
Iran University of Science and Technology（伊朗科技大学）；Sharif University of Technology（谢里夫理工大学）
Nowadays, computer-aided sperm analysis (CASA) systems have made a big leap in extracting the characteristics of spermatozoa for studies or measuring human fertility. The first step in sperm characteristics analysis is sperm detection in the frames of the video sample. In this article, we used RetinaNet, a deep fully convolutional neural network as the object detector. Sperms are small objects with few attributes, that makes the detection more difficult in high-density samples and especially when there are other particles in semen, which could be like sperm heads. One of the main attributes of…
Accurate Tracking of Arabidopsis Root Cortex Cell Nuclei in 3D Time-Lapse Microscopy Images Based on Genetic Algorithm
Graduate School of Science and Technology, Nara Institute of Science and Technology（奈良科学技术大学研究生院）；Bioresource Engineering Laboratory, Ishikawa Prefectural University（石川县立大学生物资源工程实验室）；Department of Engineering Informatics, Osaka Electro-Communication University（大阪电讯大学工程信息学系）；Exploratory Research Center on Life and Living Systems, National Institutes of Natural Sciences（国家自然科学研究院生命与生活系统探索研究中心）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
The accurate segmentation and tracking of cells in microscopy image sequences is an important task in biomedical research, e.g., for studying the development of tissues, organs or entire organisms. However, the segmentation of touching cells in images with a low signal-to-noise-ratio is still a challenging problem. In this paper, we present a method for the segmentation of touching cells in microscopy images. By using a novel representation of cell borders, inspired by distance maps, our method is capable to utilize not only touching cells but also close cells in the training process. Furtherm…
University of Copenhagen（哥本哈根大学）；Rigshospitalet（哥本哈根大学医院（Rigshospitalet））
Tracking single molecules is instrumental for quantifying the transport of molecules and nanoparticles in biological samples, e.g., in brain drug delivery studies. Existing intensity-based localisation methods are not developed for imaging with a scanning microscope, typically used for in vivo imaging. Low signal-to-noise ratios, movement of molecules out-of-focus, and high motion blur on images recorded with scanning two-photon microscopy (2PM) in vivo pose a challenge to the accurate localisation of molecules. Using data-driven models is challenging due to low data volumes, typical for in vi…
The automated analysis of microscopy images is a challenge in the context of single-cell tracking and quantification. This work has as goals the study of the performance of deep learning for segmenting microscopy images and the improvement of the previously available pipeline for tracking single cells. Deep learning techniques, mainly convolutional neural networks, have been applied to cell segmentation problems and have shown high accuracy and fast performance. To perform the image segmentation, an analysis of hyperparameters was done in order to implement a convolutional neural network with …
Vasculature is known to be of key biological significance, especially in the study of cancer. As such, considerable effort has been focused on the automated measurement and analysis of vasculature in medical and pre-clinical images. In tumors in particular, the vascular networks may be extremely irregular and the appearance of the individual vessels may not conform to classical descriptions of vascular appearance. Typically, vessels are extracted by either a segmentation and thinning pipeline, or by direct tracking. Neither of these methods are well suited to microscopy images of tumor vascula…
MicroscopyMatching: Towards a Ready-to-use Framework for Microscopy Image Analysis in Diverse Conditions
School of Computing and Communications（计算与通信学院）；Lancaster University（兰卡斯特大学）；Department of Cell Biology（细胞生物学系）；Harvard Medical School（哈佛医学院）；Department of Molecular and Cellular Biology（分子与细胞生物学系）；Harvard University（哈佛大学）
Ben-Gurion University（本-古里安大学）；The Kreitman School of Advanced Graduate Studies（克雷特曼高级研究生学院）
Live cell microscopy sequences exhibit complex spatial structures and complicated temporal behaviour, making their analysis a challenging task. Considering cell segmentation problem, which plays a significant role in the analysis, the spatial properties of the data can be captured using Convolutional Neural Networks (CNNs). Recent approaches show promising segmentation results using convolutional encoder-decoders such as the U-Net. Nevertheless, these methods are limited by their inability to incorporate temporal information, that can facilitate segmentation of individual touching cells or of …
Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；AtlasXomics Inc.（AtlasXomics 公司）；Albert-Ludwigs-University（阿尔伯特-路德维希大学）；The Johns Hopkins University（约翰斯·霍普金斯大学）
Studying cell morphology changes in time is critical to understanding cell migration mechanisms. In this work, we present a deep learning-based workflow to segment cancer cells embedded in 3D collagen matrices and imaged with phase-contrast microscopy. Our approach uses transfer learning and recurrent convolutional long-short term memory units to exploit the temporal information from the past and provide a consistent segmentation result. Lastly, we propose a geometrical-characterization approach to studying cancer cell morphology. Our approach provides stable results in time, and it is robust …
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California Nano Systems Institute (CNSI)（加州纳米系统研究所）；David Geffen School of Medicine（大卫·格芬医学院）；Hebrew University of Jerusalem（耶路撒冷希伯来大学）
Three-dimensional (3D) fluorescence microscopy in general requires axial scanning to capture images of a sample at different planes. Here we demonstrate that a deep convolutional neural network can be trained to virtually refocus a 2D fluorescence image onto user-defined 3D surfaces within the sample volume. With this data-driven computational microscopy framework, we imaged the neuron activity of a Caenorhabditis elegans worm in 3D using a time-sequence of fluorescence images acquired at a single focal plane, digitally increasing the depth-of-field of the microscope by 20-fold without any axi…
Agentic-J: An AI Agent for Biological Microscopy Image Analysis
University of Tartu（塔尔图大学）；Ruhr University Bochum（波鸿鲁尔大学）；University of Amsterdam（阿姆斯特丹大学）；University of Notre Dame（圣母大学）；University of Duisburg-Essen（杜伊斯堡-埃森大学）
3D fluorescence microscopy of living organisms has increasingly become an essential and powerful tool in biomedical research and diagnosis. An exploding amount of imaging data has been collected, whereas efficient and effective computational tools to extract information from them are still lagging behind. This is largely due to the challenges in analyzing biological data. Interesting biological structures are not only small, but are often morphologically irregular and highly dynamic. Although tracking cells in live organisms has been studied for years, existing tracking methods for cells are n…
Ultrasound localization microscopy (ULM) refers to a promising medical imaging modality that systematically leverages the advantages of contrast-enhanced ultrasound (CEUS) to surpass the diffraction barrier and delineate the microvascular map. Localization and tracking of microbubbles (MBs), two significant steps of ULM, facilitate generating the vascular map and the velocity distribution, respectively. Herein, we propose a novel MB tracking technique considering temporal pairing as a bubble-set registration problem. Iterative registration is performed between the bubble sets in two consecutiv…
HHMI Janelia Research Campus（霍华德·休斯医学研究所珍利亚研究园区）；Institute of Neuroinformatics UZH/ETHZ（苏黎世大学与苏黎世联邦理工学院神经信息学研究所）
We present a method for microtubule tracking in electron microscopy volumes. Our method first identifies a sparse set of voxels that likely belong to microtubules. Similar to prior work, we then enumerate potential edges between these voxels, which we represent in a candidate graph. Tracks of microtubules are found by selecting nodes and edges in the candidate graph by solving a constrained optimization problem incorporating biological priors on microtubule structure. For this, we present a novel integer linear programming formulation, which results in speed-ups of three orders of magnitude an…
Laboratoire Jean-Perrin UMR 8237, Sorbonne Université（索邦大学让-佩兰实验室）；SABILAb；CMAP, Ecole Polytechnique, Université Paris-Saclay（巴黎萨克雷大学综合理工学院应用数学中心）；Heuritech
The mother machine is a popular microfluidic device that allows long-term time-lapse imaging of thousands of cells in parallel by microscopy. It has become a valuable tool for single-cell level quantitative analysis and characterization of many cellular processes such as gene expression and regulation, mutagenesis or response to antibiotics. The automated and quantitative analysis of the massive amount of data generated by such experiments is now the limiting step. In particular the segmentation and tracking of bacteria cells imaged in phase-contrast microscopy---with error rates compatible wi…
Automatic analysis of spatio-temporal microscopy images is inevitable for state-of-the-art research in the life sciences. Recent developments in deep learning provide powerful tools for automatic analyses of such image data, but heavily depend on the amount and quality of provided training data to perform well. To this end, we developed a new method for realistic generation of synthetic 2D+t microscopy image data of fluorescently labeled cellular nuclei. The method combines spatiotemporal statistical shape models of different cell cycle stages with a conditional GAN to generate time series of …
Cell Behavior Video Classification Challenge, a benchmark for computer vision methods in time-lapse microscopy
Euler Institute, Faculty of Informatics, Università della Svizzera italiana（欧拉研究所，信息学院，瑞士大学）；International Center for Advanced Computing in Medicine (ICAM), University of Pavia（国际医学先进计算中心（ICAM），帕维亚大学）；Imaging Platform, ACRF INCITe Centre, Garvan Institute of Medical Research（成像平台，ACRF INCITe中心，嘉文医学研究所）；Tacoma School of Engineering & Technology, University of Washington（塔科马工程与技术学院，华盛顿大学）；Data Science, Institute for Computing and Information Sciences, Radboud University（数据科学，计算与信息科学研究所，拉德堡德大学）；Imaging Core, Lerner Research Institute, Cleveland Clinic（成像核心，勒纳研究研究所，克利夫兰诊所）；Institute of Biomedical Engineering, University of Toronto（生物医学工程研究所，多伦多大学）；Center for Research in Computer Vision, University of Central Florida（计算机视觉研究中心，佛罗里达大学）；Computational Biology Group, Data Science Platform, Garvan Institute of Medical Research（计算生物学小组，数据科学平台，嘉文医学研究所）
Deep Joint Denoising and Detection for Enhanced Intracellular Particle Analysis
Erasmus University Medical Center（伊拉斯姆斯大学医学中心）；Utrecht University（乌得勒支大学）；University of New South Wales (UNSW)（新南威尔士大学）；NVIDIA Corporation（英伟达公司）；SURFsara
Localization microscopy is an imaging technique in which the positions of individual nanoscale point emitters (e.g. fluorescent molecules) are determined at high precision from their images. This is the key ingredient in single/multiple-particle-tracking and several super-resolution microscopy approaches. Localization in three-dimensions (3D) can be performed by modifying the image that a point-source creates on the camera, namely, the point-spread function (PSF). The PSF is engineered using additional optical elements to vary distinctively with the depth of the point-source. However, localizi…
In this work, we use the standard deviation of image pixel intensity to analyse the speed, direction and surface-interaction of microparticles in fluid. First, we present an analytical model for estimating the total variance in the image space for directed or diffusive motion of microparticles and show that this measure is correlated to the density and speed of the particles. The analytical model was found to have good agreement with numerical simulations for low particle density. Then, using only the local image variance we obtain the magnitude and direction of the particle velocity in a rect…
Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scienti…
