The ability to automatically detect, classify, calculate the size, number, and grade of retinal cells and other biological objects is critically important in eye disease like age-related macular degeneration (AMD). In this paper, we developed an automated tool based on deep learning technique and Mask R-CNN model to analyze large datasets of transmission electron microscopy (TEM) images and quantify retinal cells with high speed and precision. We considered three categories for outer nuclear layer (ONL) cells: live, intermediate, and pyknotic. We trained the model using a dataset of 24 samples. We then optimized the hyper-parameters using another set of 6 samples. The results of this research, after applying to the test datasets, demonstrated that our method is highly accurate for automatically detecting, categorizing, and counting cell nuclei in the ONL of the retina. Performance of our model was tested using general metrics: general mean average precision (mAP) for detection; and precision, recall, F1-score, and accuracy for categorizing and counting.
Segmentation-free integration of nuclei morphology and spatial transcriptomics for retinal images
Dept. of Information Technology（信息技术系）；Dept. of Molecular, Cell and Developmental Biology（分子、细胞与发育生物学系）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
University of Warwick（华威大学）；University Hospitals Coventry and Warwickshire（考文垂和华威郡大学医院）
Nuclear segmentation, classification and quantification within Haematoxylin & Eosin stained histology images enables the extraction of interpretable cell-based features that can be used in downstream explainable models in computational pathology (CPath). However, automatic recognition of different nuclei is faced with a major challenge in that there are several different types of nuclei, some of them exhibiting large intra-class variability. To help drive forward research and innovation for automatic nuclei recognition in CPath, we organise the Colon Nuclei Identification and Counting (CoNIC) …
The College of New Jersey（新泽西学院）；University of Oxford（牛津大学）；Ludwig Institute for Cancer Research（路德维希癌症研究所）
Disease complications can alter vascular network morphology and disrupt tissue functioning. Diabetic retinopathy, for example, is a complication of types 1 and 2 diabetes mellitus that can cause blindness. Microvascular diseases are assessed by visual inspection of retinal images, but this can be challenging when diseases exhibit silent symptoms or patients cannot attend in-person meetings. We examine the performance of machine learning algorithms in detecting microvascular disease when trained on statistical and topological summaries of segmented retinal vascular images. We apply our methods …
University of Cambridge（剑桥大学）；Chinese University of Hong Kong（香港中文大学）
We address the problem of automated nuclear segmentation, classification, and quantification from Haematoxylin and Eosin stained histology images, which is of great relevance for several downstream computational pathology applications. In this work, we present a solution framed as a simultaneous semantic and instance segmentation framework. Our solution is part of the Colon Nuclei Identification and Counting (CoNIC) Challenge. We first train a semantic and instance segmentation model separately. Our framework uses as backbone HoverNet and Cascade Mask-RCNN models. We then ensemble the results …
Chonnam National University（全南国立大学）；Chonnam National University Medical School（全南国立大学医学院）
Nuclei Identification and Counting is the most important morphological feature of cancers, especially in the colon. Many deep learning-based methods have been proposed to deal with this problem. In this work, we construct an extension of Hovernet for nuclei identification and counting to address the problem named MF-Hovernet. Our proposed model is the combination of multiple filer block to Hovernet architecture. The current result shows the efficiency of multiple filter block to improve the performance of the original Hovernet model.
Nuclear segmentation, classification and quantification within Haematoxylin & Eosin stained histology images enables the extraction of interpretable cell-based features that can be used in downstream explainable models in computational pathology (CPath). However, automatic recognition of different nuclei is faced with a major challenge in that there are several different types of nuclei, some of them exhibiting large intraclass variability. In this work, we propose an approach that combine Separable-HoverNet and Instance-YOLOv5 to indentify colon nuclei small and unbalanced. Our approach can a…
Queen Mary University of London（伦敦玛丽女王大学）；University of Leicester（莱斯特大学）
The detection of nuclei and cells in histology images is of great value in both clinical practice and pathological studies. However, multiple reasons such as morphological variations of nuclei or cells make it a challenging task where conventional object detection methods cannot obtain satisfactory performance in many cases. A detection task consists of two sub-tasks, classification and localization. Under the condition of dense object detection, classification is a key to boost the detection performance. Considering this, we propose similarity based region proposal networks (SRPN) for nuclei …
In this paper we propose a technique to adapt a convolutional neural network (CNN) based object counter to additional visual domains and object types while still preserving the original counting function. Domain-specific normalisation and scaling operators are trained to allow the model to adjust to the statistical distributions of the various visual domains. The developed adaptation technique is used to produce a singular patch-based counting regressor capable of counting various object types including people, vehicles, cell nuclei and wildlife. As part of this study a challenging new cell co…
Guangdong Provincial People’s Hospital, Guangdong Academy of Medical Sciences（广东省人民医院、广东省医学科学院）；Guilin University of Electronic Technology（桂林电子科技大学）
Nuclear segmentation and classification is an essential step for computational pathology. TIA lab from Warwick University organized a nuclear segmentation and classification challenge (CoNIC) for H&E stained histopathology images in colorectal cancer with two highly correlated tasks, nuclei segmentation and classification task and cellular composition task. There are a few obstacles we have to address in this challenge, 1) limited training samples, 2) color variation, 3) imbalanced annotations, 4) similar morphological appearance among classes. To deal with these challenges, we proposed a stan…
University of Warwick（华威大学）；Tarbiat Modares University（塔比亚特·莫达雷斯大学）；University of Sheffield（谢菲尔德大学）；Alan Turing Institute（艾伦·图灵研究所）
Nuclei detection is an important task in the histology domain as it is a main step toward further analysis such as cell counting, cell segmentation, study of cell connections, etc. This is a challenging task due to the complex texture of histology image, variation in shape, and touching cells. To tackle these hurdles, many approaches have been proposed in the literature where deep learning methods stand on top in terms of performance. Hence, in this paper, we propose a novel framework for nuclei detection based on Mixture Density Networks (MDNs). These networks are suitable to map a single inp…
Muroran Institute of Technology（室兰工业大学）；Niigata University of Healthcare and Welfare（新泻医疗福祉大学）
Nuclear segmentation, classification and quantification within Haematoxylin & Eosin stained histology images enables the extraction of interpretable cell-based features that can be used in downstream explainable models in computational pathology. The Colon Nuclei Identification and Counting (CoNIC) Challenge is held to help drive forward research and innovation for automatic nuclei recognition in computational pathology. This report describes our proposed method submitted to the CoNIC challenge. Our method employs a multi-task learning framework, which performs a panoptic segmentation task and…
Iran University of Science and Technology（伊朗科技大学）；Sharif University of Technology（谢里夫理工大学）
Pistachios are nutritious nuts that are sorted based on the shape of their shell into two categories: Open-mouth and Closed-mouth. The open-mouth pistachios are higher in price, value, and demand than the closed-mouth pistachios. Because of these differences, it is considerable for production companies to precisely count the number of each kind. This paper aims to propose a new system for counting the different types of pistachios with computer vision. We have introduced and shared a new dataset of pistachios, including six videos with a total length of 167 seconds and 3927 labeled pistachios.…
Two major causes of death in the United States and worldwide are stroke and myocardial infarction. The underlying cause of both is thrombi released from ruptured or eroded unstable atherosclerotic plaques that occlude vessels in the heart (myocardial infarction) or the brain (stroke). Clinical studies show that plaque composition plays a more important role than lesion size in plaque rupture or erosion events. To determine the plaque composition, various cell types in 3D cardiovascular immunofluorescent images of plaque lesions are counted. However, counting these cells manually is expensive, …
Monitoring of colonial waterbird nesting islands is essential to tracking waterbird population trends, which are used for evaluating ecosystem health and informing conservation management decisions. Recently, unmanned aerial vehicles, or drones, have emerged as a viable technology to precisely monitor waterbird colonies. However, manually counting waterbirds from hundreds, or potentially thousands, of aerial images is both difficult and time-consuming. In this work, we present a deep learning pipeline that can be used to precisely detect, count, and monitor waterbirds using aerial imagery coll…
Automated cell nucleus segmentation and classification are required to assist pathologists in their decision making. The Colon Nuclei Identification and Counting Challenge 2022 (CoNIC Challenge 2022) supports the development and comparability of segmentation and classification methods for histopathological images. In this contribution, we describe our CoNIC Challenge 2022 method ciscNet to segment, classify and count cell nuclei, and report preliminary evaluation results. Our code is available at https://git.scc.kit.edu/ciscnet/ciscnet-conic-2022.
Dr. A.P.J. Abdul Kalam Technical University（阿卜杜勒·卡拉姆技术大学）
Every year millions of people die due to disease of Cancer. Due to its invasive nature it is very complex to cure even in primary stages. Hence, only method to survive this disease completely is via forecasting by analyzing the early mutation in cells of the patient biopsy. Cell Segmentation can be used to find cell which have left their nuclei. This enables faster cure and high rate of survival. Cell counting is a hard, yet tedious task that would greatly benefit from automation. To accomplish this task, segmentation of cells need to be accurate. In this paper, we have improved the learning o…
FEMTO-ST institute（弗朗什-孔泰电子、机械、热学与光学科学与技术研究所）；Univ. Bourgogne Franche-Comte（勃艮第-弗朗什-孔泰大学）；CNRS（法国国家科学研究中心）；ENSMM（国立高等机械与微技术学院）；Beihang University（北京航空航天大学）；Hôpital Nord Franche-Comté（北弗朗什-孔泰医院）；Batna2 University（巴特纳第二大学）
The Ki-67 proliferation index is an essential biomarker that helps pathologists to diagnose and select appropriate treatments. However, automatic evaluation of Ki-67 is difficult due to nuclei overlapping and complex variations in their properties. This paper proposes an integrated pipeline for accurate automatic counting of Ki-67, where the impact of nuclei separation techniques is highlighted. First, semantic segmentation is performed by combining the Squeez and Excitation Resnet and Unet algorithms to extract nuclei from the background. The extracted nuclei are then divided into overlapped …
Keep It Accurate and Robust: An Enhanced Nuclei Analysis Framework
Institute of Artificial Intelligence, Shanghai University（上海大学人工智能研究院）；Stanford University School of Medicine（斯坦福大学医学院）；Tencent AI Lab（腾讯AI实验室）；Shanghai Aitrox Technology Corporation Limited（上海艾特瑞克斯科技有限公司）；The Affiliated Yantai Yuhuangding Hospital of Qingdao University（青岛大学附属烟台毓璜顶医院）
Circulating Tumor Cells (CTCs) bear great promise as biomarkers in tumor prognosis. However, the process of identification and later enumeration of CTCs require manual labor, which is error-prone and time-consuming. The recent developments in object detection via Deep Learning using Mask-RCNNs and wider availability of pre-trained models have enabled sensitive tasks with limited data of such to be tackled with unprecedented accuracy. In this report, we present a novel 3-stage detection model for automated identification of Circulating Tumor Cells in multi-channel darkfield microscopic images c…
A Survey on Cell Nuclei Instance Segmentation and Classification: Leveraging Context and Attention
INESC TEC - Institute for Systems and Computer Engineering, Technology and Science（INESC TEC系统与计算机工程、技术与科学研究所）；University of Porto - Faculty of Engineering（波尔图大学工程学院）；IMP Diagnostics（IMP诊断公司）；Portuguese Oncology Institute of Porto (IPO Porto)（波尔图葡萄牙肿瘤研究所）；University of Porto (ICBAS-UP)（波尔图大学（ICBAS-UP））；FCTUC - Faculty of Science and Technology, University of Coimbra（科英布拉大学科学与技术学院（FCTUC））
University of Western São Paulo (UNOESTE)（西圣保罗大学）；Federal University of Mato Grosso do Sul (UFMS)（南马托格罗索联邦大学）；Brazilian Agricultural Research Agency (EMBRAPA)（巴西农业研究局）；University of Santa Catarina State (UDESC)（圣卡塔琳娜州立大学）
In this paper, we propose a novel deep learning method based on a Convolutional Neural Network (CNN) that simultaneously detects and geolocates plantation-rows while counting its plants considering highly-dense plantation configurations. The experimental setup was evaluated in a cornfield with different growth stages and in a Citrus orchard. Both datasets characterize different plant density scenarios, locations, types of crops, sensors, and dates. A two-branch architecture was implemented in our CNN method, where the information obtained within the plantation-row is updated into the plant det…
3D Segmentation of Neuronal Nuclei and Cell-Type Identification using Multi-channel Information
Universidad Politécnica de Madrid（马德里理工大学）；Center for Computational Simulation（计算模拟中心）；Instituto Cajal (CSIC)（卡哈尔研究所（西班牙国家研究委员会））；Laboratorio Cajal de Circuitos Corticales (CTB)（卡哈尔皮层回路实验室（CTB））
NEC Laboratories America（NEC美国实验室）；NEC Corporation（NEC公司）；Hokkaido University Hospital（北海道大学医院）
We propose a method to accurately obtain the ratio of tumor cells over an entire histological slide. We use deep fully convolutional neural network models trained to detect and classify cells on images of H&E-stained tissue sections. Pathologists' labels consisting of exhaustive nuclei locations and tumor regions were used to trained the model in a supervised fashion. We show that combining two models, each working at a different magnification allows the system to capture both cell-level details and surrounding context to enable successful detection and classification of cells as either tumor-…
University of Warwick（华威大学）；Histofy Ltd（Histofy有限公司）；EPFL（洛桑联邦理工学院）；The University of Hong Kong（香港大学）；Tencent AI Lab（腾讯AI实验室）；Sichuan University（四川大学）；Tsinghua University（清华大学）；Max-Delbrueck-Center for Molecular Medicine in the Helmholtz Association（亥姆霍兹协会马克斯·德尔布吕克分子医学中心）；Humboldt University of Berlin（柏林洪堡大学）；Charité University Medicine（柏林夏里特医学院）；University of Bern（伯尔尼大学）；University of Cambridge（剑桥大学）；Chinese University of Hong Kong（香港中文大学）；Arontier Co. Ltd（Arontier有限公司）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；CNRS（法国国家科学研究中心）；IFCE（法国马与骑术研究所）；INRAE（法国国家农业、食品与环境研究院）；Université de Tours（图尔大学）；Institut de Génétique et de Biologie Moléculaire et Cellulaire（分子与细胞遗传学与生物学研究所）；Centre National de la Recherche Scientifique（法国国家科学研究中心）；Institut National de la Santé et de la Recherche Médicale（法国国家健康与医学研究院）；Université de Strasbourg（斯特拉斯堡大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；South China University of Technology（华南理工大学）；University Hospitals Coventry and Warwickshire NHS Trust（考文垂与沃里克郡大学医院NHS信托）
Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 wh…
Fluorescent Neuronal Cells v2: Multi-Task, Multi-Format Annotations for Deep Learning in Microscopy
National Institute of Nuclear Physics（国家核物理研究所）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；University of Bologna（博洛尼亚大学）
Interpretable histopathology-based prediction of disease relevant features in Inflammatory Bowel Disease biopsies using weakly-supervised deep learning
