Quantitative analysis of the dynamics of tiny cellular and sub-cellular structures, known as particles, in time-lapse cell microscopy sequences requires the development of a reliable multi-target tracking method capable of tracking numerous similar targets in the presence of high levels of noise, high target density, complex motion patterns and intricate interactions. In this paper, we propose a framework for tracking these structures based on the random finite set Bayesian filtering framework. We focus on challenging biological applications where image characteristics such as noise and background intensity change during the acquisition process. Under these conditions, detection methods usually fail to detect all particles and are often followed by missed detections and many spurious measurements with unknown and time-varying rates. To deal with this, we propose a bootstrap filter composed of an estimator and a tracker. The estimator adaptively estimates the required meta parameters for the tracker such as clutter rate and the detection probability of the targets, while the tracker estimates the state of the targets. Our results show that the proposed approach can outperform state-of-the-art particle trackers on both synthetic and real data in this regime.
Istituto Sistemi e Electronica Applicata (ISEA)（应用系统与电子研究所）；Scuola universitaria professionale della Svizzera italiana (SUPSI)（瑞士意大利语区应用科技大学）；Grupo de investigacion SISTEMIC（SISTEMIC研究组）；Facultad de Ingenieria, Universidad de Antioquia UdeA（安蒂奥基亚大学工程学院）
An automatic approach to counting any kind of cells could alleviate work of the experts and boost the research in fields such as regenerative medicine. In this paper, a method for microscopy cell counting using multiple frames (hence temporal information) is proposed. Unlike previous approaches where the cell counting is done independently in each frame (static cell counting), in this work the cell counting prediction is done using multiple frames (dynamic cell counting). A spatiotemporal model using ConvNets and long short term memory (LSTM) recurrent neural networks is proposed to overcome t…
Time-lapse fluorescent microscopy (TLFM) combined with predictive mathematical modelling is a powerful tool to study the inherently dynamic processes of life on the single-cell level. Such experiments are costly, complex and labour intensive. A complimentary approach and a step towards in silico experimentation, is to synthesise the imagery itself. Here, we propose Multi-StyleGAN as a descriptive approach to simulate time-lapse fluorescence microscopy imagery of living cells, based on a past experiment. This novel generative adversarial network synthesises a multi-domain sequence of consecutiv…
Accurate Tracking of Arabidopsis Root Cortex Cell Nuclei in 3D Time-Lapse Microscopy Images Based on Genetic Algorithm
Graduate School of Science and Technology, Nara Institute of Science and Technology（奈良科学技术大学研究生院）；Bioresource Engineering Laboratory, Ishikawa Prefectural University（石川县立大学生物资源工程实验室）；Department of Engineering Informatics, Osaka Electro-Communication University（大阪电讯大学工程信息学系）；Exploratory Research Center on Life and Living Systems, National Institutes of Natural Sciences（国家自然科学研究院生命与生活系统探索研究中心）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
University of Bologna（博洛尼亚大学）；University of Tor Vergata（罗马第二大学（托尔·韦尔加塔大学））；Institute Curie Centre de Recherche（居里研究所研究中心）；Paris Sciences et Lettres Research University（巴黎文理研究大学）
Biological experiments based on organ-on-chips (OOCs) exploit light Time-Lapse Microscopy (TLM) for a direct observation of cell movement that is an observable signature of underlying biological processes. A high spatial resolution is essential to capture cell dynamics and interactions from recorded experiments by TLM. Unfortunately, due to physical and cost limitations, acquiring high resolution videos is not always possible. To overcome the problem, we present here a new deep learning-based algorithm that extends the well known Deep Image Prior (DIP) to TLM Video Super Resolution (SR) withou…
University of California Santa Barbara（加州大学圣巴巴拉分校）
We propose a novel weakly supervised method to improve the boundary of the 3D segmented nuclei utilizing an over-segmented image. This is motivated by the observation that current state-of-the-art deep learning methods do not result in accurate boundaries when the training data is weakly annotated. Towards this, a 3D U-Net is trained to get the centroid of the nuclei and integrated with a simple linear iterative clustering (SLIC) supervoxel algorithm that provides better adherence to cluster boundaries. To track these segmented nuclei, our algorithm utilizes the relative nuclei location depict…
Cell detection is the task of detecting the approximate positions of cell centroids from microscopy images. Recently, convolutional neural network-based approaches have achieved promising performance. However, these methods require a certain amount of annotation for each imaging condition. This annotation is a time-consuming and labor-intensive task. To overcome this problem, we propose a semi-supervised cell-detection method that effectively uses a time-lapse sequence with one labeled image and the other images unlabeled. First, we train a cell-detection network with a one-labeled image and e…
Cell Behavior Video Classification Challenge, a benchmark for computer vision methods in time-lapse microscopy
Euler Institute, Faculty of Informatics, Università della Svizzera italiana（欧拉研究所，信息学院，瑞士大学）；International Center for Advanced Computing in Medicine (ICAM), University of Pavia（国际医学先进计算中心（ICAM），帕维亚大学）；Imaging Platform, ACRF INCITe Centre, Garvan Institute of Medical Research（成像平台，ACRF INCITe中心，嘉文医学研究所）；Tacoma School of Engineering & Technology, University of Washington（塔科马工程与技术学院，华盛顿大学）；Data Science, Institute for Computing and Information Sciences, Radboud University（数据科学，计算与信息科学研究所，拉德堡德大学）；Imaging Core, Lerner Research Institute, Cleveland Clinic（成像核心，勒纳研究研究所，克利夫兰诊所）；Institute of Biomedical Engineering, University of Toronto（生物医学工程研究所，多伦多大学）；Center for Research in Computer Vision, University of Central Florida（计算机视觉研究中心，佛罗里达大学）；Computational Biology Group, Data Science Platform, Garvan Institute of Medical Research（计算生物学小组，数据科学平台，嘉文医学研究所）
Kymographs are widely used to represent and anal- yse spatio-temporal dynamics of fluorescence markers along curvilinear biological compartments. These objects have a sin- gular geometry, thus kymograph reconstruction is inherently an analog image processing task. However, the existing approaches are essentially digital: the kymograph photometry is sampled directly from the time-lapse images. As a result, such kymographs rely on raw image data that suffer from the degradations entailed by the image formation process and the spatio-temporal resolution of the imaging setup. In this work, we addr…
Time-lapse fluorescence microscopy (TLFM) is an important and powerful tool in synthetic biological research. Modeling TLFM experiments based on real data may enable researchers to repeat certain experiments with minor effort. This thesis is a study towards deep learning-based modeling of TLFM experiments on the image level. The modeling of TLFM experiments, by way of the example of trapped yeast cells, is split into two tasks. The first task is to generate synthetic image data based on real image data. To approach this problem, a novel generative adversarial network, for conditionalized and u…
We present a novel graph neural network (GNN) approach for cell tracking in high-throughput microscopy videos. By modeling the entire time-lapse sequence as a direct graph where cell instances are represented by its nodes and their associations by its edges, we extract the entire set of cell trajectories by looking for the maximal paths in the graph. This is accomplished by several key contributions incorporated into an end-to-end deep learning framework. We exploit a deep metric learning algorithm to extract cell feature vectors that distinguish between instances of different biological cells…
Deep Neural Networks are increasingly used in video frame interpolation tasks such as frame rate changes as well as generating fake face videos. Our project aims to apply recent advances in Deep video interpolation to increase the temporal resolution of fluorescent microscopy time-lapse movies. To our knowledge, there is no previous work that uses Convolutional Neural Networks (CNN) to generate frames between two consecutive microscopy images. We propose a fully convolutional autoencoder network that takes as input two images and generates upto seven intermediate images. Our architecture has t…
Department of Molecular and Cellular Biology（分子与细胞生物学系）；School of Engineering and Applied Sciences（工程与应用科学学院）；Tel Aviv Sourasky Medical Center（特拉维夫苏拉斯基医疗中心）；Harvard Graduate Program in Biophysics（哈佛大学生物物理学研究生项目）；Harvard University（哈佛大学）
A major challenge in clinical In-Vitro Fertilization (IVF) is selecting the highest quality embryo to transfer to the patient in the hopes of achieving a pregnancy. Time-lapse microscopy provides clinicians with a wealth of information for selecting embryos. However, the resulting movies of embryos are currently analyzed manually, which is time consuming and subjective. Here, we automate feature extraction of time-lapse microscopy of human embryos with a machine-learning pipeline of five convolutional neural networks (CNNs). Our pipeline consists of (1) semantic segmentation of the regions of …
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California NanoSystems Institute（加州纳米系统研究所）；Istanbul University（伊斯坦布尔大学）；David Geffen School of Medicine（大卫·格芬医学院）
We present a computational live bacteria detection system that periodically captures coherent microscopy images of bacterial growth inside a 60 mm diameter agar-plate and analyzes these time-lapsed holograms using deep neural networks for rapid detection of bacterial growth and classification of the corresponding species. The performance of our system was demonstrated by rapid detection of Escherichia coli and total coliform bacteria (i.e., Klebsiella aerogenes and Klebsiella pneumoniae subsp. pneumoniae) in water samples. These results were confirmed against gold-standard culture-based result…
Slovak University of Technology in Bratislava（布拉迪斯拉发斯洛伐克理工大学）；CNRS（法国国家科学研究中心）；Univ. Montpellier（蒙彼利埃大学）
The automated segmentation and tracking of macrophages during their migration are challenging tasks due to their dynamically changing shapes and motions. This paper proposes a new algorithm to achieve automatic cell tracking in time-lapse microscopy macrophage data. First, we design a segmentation method employing space-time filtering, local Otsu's thresholding, and the SUBSURF (subjective surface segmentation) method. Next, the partial trajectories for cells overlapping in the temporal direction are extracted in the segmented images. Finally, the extracted trajectories are linked by consideri…
Unsupervised cell segmentation by fast Gaussian Processes
Department of Statistics and Applied Probability, University of California, Santa Barbara, CA（统计与应用概率系，加州大学圣芭芭拉分校）；Department of Electrical and Computer Engineering, University of California, Santa Barbara, CA（电气与计算机工程系，加州大学圣芭芭拉分校）
Deep Joint Denoising and Detection for Enhanced Intracellular Particle Analysis
Erasmus University Medical Center（伊拉斯姆斯大学医学中心）；Utrecht University（乌得勒支大学）；University of New South Wales (UNSW)（新南威尔士大学）；NVIDIA Corporation（英伟达公司）；SURFsara
Laboratoire Jean-Perrin UMR 8237, Sorbonne Université（索邦大学让-佩兰实验室）；SABILAb；CMAP, Ecole Polytechnique, Université Paris-Saclay（巴黎萨克雷大学综合理工学院应用数学中心）；Heuritech
The mother machine is a popular microfluidic device that allows long-term time-lapse imaging of thousands of cells in parallel by microscopy. It has become a valuable tool for single-cell level quantitative analysis and characterization of many cellular processes such as gene expression and regulation, mutagenesis or response to antibiotics. The automated and quantitative analysis of the massive amount of data generated by such experiments is now the limiting step. In particular the segmentation and tracking of bacteria cells imaged in phase-contrast microscopy---with error rates compatible wi…
In this paper, we propose a hybrid approach for multi-object microbial cell segmentation. The approach combines an ML-based detection with a geometry-aware variational-based segmentation using B-splines that are parametrized based on a geometric model of the cell shape. The detection is done first using YOLOv5. In a second step, each detected cell is segmented individually. Thus, the segmentation only needs to be done on a per-cell basis, which makes it amenable to a variational approach that incorporates prior knowledge on the geometry. Here, the contour of the segmentation is modelled as clo…
TransientTrack: Advanced Multi-Object Tracking and Classification of Cancer Cells with Transient Fluorescent Signals
Institute for Biomedical Informatics, Faculty of Medicine；University Hospital Cologne, University of Cologne, Cologne, Germany.；Faculty of Mathematics；Natural Sciences, University of Cologne, Cologne, Germany.；Center for Molecular Medicine Cologne (CMMC), Faculty of Medicine；Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, Germany；Charité Comprehensive Cancer Center, Charité Universitätsmedizin, Berlin, Germany.；Centre for Genomic Regulation, Barcelona, Spain.
PhagoStat a scalable and interpretable end to end framework for efficient quantification of cell phagocytosis in neurodegenerative disease studies
Sorbonne University（索邦大学）；CNRS（法国国家科学研究中心）；Inserm（法国国家健康与医学研究院）；AP-HP（巴黎公共医疗集团）；Inria（法国国家信息与自动化研究所）；Paris Brain Institute - ICM（巴黎大脑研究所 - ICM）；PSL Research university（巴黎文理研究大学）；EPHE（高等研究实践学院）
