Weakly-supervised learning (WSL) has recently triggered substantial interest as it mitigates the lack of pixel-wise annotations. Given global image labels, WSL methods yield pixel-level predictions (segmentations), which enable to interpret class predictions. Despite their recent success, mostly with natural images, such methods can face important challenges when the foreground and background regions have similar visual cues, yielding high false-positive rates in segmentations, as is the case in challenging histology images. WSL training is commonly driven by standard classification losses, which implicitly maximize model confidence, and locate the discriminative regions linked to classification decisions. Therefore, they lack mechanisms for modeling explicitly non-discriminative regions and reducing false-positive rates. We propose novel regularization terms, which enable the model to seek both non-discriminative and discriminative regions, while discouraging unbalanced segmentations. We introduce high uncertainty as a criterion to localize non-discriminative regions that do not affect classifier decision, and describe it with original Kullback-Leibler (KL) divergence losses evaluating the deviation of posterior predictions from the uniform distribution. Our KL terms encourage high uncertainty of the model when the latter inputs the latent non-discriminative regions. Our loss integrates: (i) a cross-entropy seeking a foreground, where model confidence about class prediction is high; (ii) a KL regularizer seeking a background, where model uncertainty is high; and (iii) log-barrier terms discouraging unbalanced segmentations. Comprehensive experiments and ablation studies over the public GlaS colon cancer data and a Camelyon16 patch-based benchmark for breast cancer show substantial improvements over state-of-the-art WSL methods, and confirm the effect of our new regularizers.
The Hong Kong University of Science and Technology（香港科技大学）；HKUST Shenzhen Research Institute（香港科技大学深圳研究院）
Developing an AI-assisted gland segmentation method from histology images is critical for automatic cancer diagnosis and prognosis; however, the high cost of pixel-level annotations hinders its applications to broader diseases. Existing weakly-supervised semantic segmentation methods in computer vision achieve degenerative results for gland segmentation, since the characteristics and problems of glandular datasets are different from general object datasets. We observe that, unlike natural images, the key problem with histology images is the confusion of classes owning to morphological homogene…
Polytechnique Montréal（蒙特利尔理工学院）；Télécom Paristech University（巴黎电信学院）；Montréal Heart Institute（蒙特利尔心脏研究所）；Université de Montréal（蒙特利尔大学）
Semantic segmentation is a crucial task in biomedical image processing, which recent breakthroughs in deep learning have allowed to improve. However, deep learning methods in general are not yet widely used in practice since they require large amount of data for training complex models. This is particularly challenging for biomedical images, because data and ground truths are a scarce resource. Annotation efforts for biomedical images come with a real cost, since experts have to manually label images at pixel-level on samples usually containing many instances of the target anatomy (e.g. in his…
University of Warwick（华威大学）；The Alan Turing Institute（阿兰·图灵研究所）；University Hospitals Coventry & Warwickshire（考文垂和沃里克郡大学医院）；Histofy Ltd（Histofy有限公司）
Automated synthesis of histology images has several potential applications in computational pathology. However, no existing method can generate realistic tissue images with a bespoke cellular layout or user-defined histology parameters. In this work, we propose a novel framework called SynCLay (Synthesis from Cellular Layouts) that can construct realistic and high-quality histology images from user-defined cellular layouts along with annotated cellular boundaries. Tissue image generation based on bespoke cellular layouts through the proposed framework allows users to generate different histolo…
University of Warwick（华威大学）；University Hospitals Coventry and Warwickshire（考文垂和沃里克大学医院）；The Alan Turing Institute（艾伦·图灵研究所）
Synthetic images can be used for the development and evaluation of deep learning algorithms in the context of limited availability of data. In the field of computational pathology, where histology images are large in size and visual context is crucial, synthesis of large high resolution images via generative modeling is a challenging task. This is due to memory and computational constraints hindering the generation of large images. To address this challenge, we propose a novel SAFRON (Stitching Across the FRONtiers) framework to construct realistic, large high resolution tissue image tiles fro…
University of Warwick（华威大学）；The Chinese University of Hong Kong（香港中文大学）；University Hospitals Coventry and Warwickshire（考文垂与华威大学医院）；The Alan Turing Institute（艾伦·图灵研究所）
The analysis of glandular morphology within colon histopathology images is an important step in determining the grade of colon cancer. Despite the importance of this task, manual segmentation is laborious, time-consuming and can suffer from subjectivity among pathologists. The rise of computational pathology has led to the development of automated methods for gland segmentation that aim to overcome the challenges of manual segmentation. However, this task is non-trivial due to the large variability in glandular appearance and the difficulty in differentiating between certain glandular and non-…
Tissue Image Analytics Centre（组织图像分析中心）；University of Warwick（华威大学）
Semantic segmentation of various tissue and nuclei types in histology images is fundamental to many downstream tasks in the area of computational pathology (CPath). In recent years, Deep Learning (DL) methods have been shown to perform well on segmentation tasks but DL methods generally require a large amount of pixel-wise annotated data. Pixel-wise annotation sometimes requires expert's knowledge and time which is laborious and costly to obtain. In this paper, we present a consistency based semi-supervised learning (SSL) approach that can help mitigate this challenge by exploiting a large amo…
Arizona State University（亚利桑那州立大学）；Barrow Neurological Institute（巴罗神经学研究所）；St. Joseph’s Hospital and Medical Center（圣约瑟夫医院和医疗中心）
Confocal laser endomicroscopy (CLE) allow on-the-fly in vivo intraoperative imaging in a discreet field of view, especially for brain tumors, rather than extracting tissue for examination ex vivo with conventional light microscopy. Fluorescein sodium-driven CLE imaging is more interactive, rapid, and portable than conventional hematoxylin and eosin (H&E)-staining. However, it has several limitations: CLE images may be contaminated with artifacts (motion, red blood cells, noise), and neuropathologists are mainly trained on colorful stained histology slides like H&E while the CLE images are gray…
Improving Performance in Colorectal Cancer Histology Decomposition using Deep and Ensemble Machine Learning
University of Jyväskylä（于韦斯屈莱大学）；University of Helsinki（赫尔辛基大学）；University of Turku（图尔库大学）；Turku University Hospital（图尔库大学医院）；Hospital Nova of Central Finland（中芬兰诺瓦医院）
A Generative Foundation Model for Multimodal Histopathology
Department of Radiation Oncology, Stanford University School of Medicine（斯坦福大学医学院放射肿瘤学系）；Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine（斯坦福大学医学院精神病学与行为科学系）；Department of Statistics and Data Science, Yale University（耶鲁大学统计与数据科学系）；Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Department of Electrical Engineering, Stanford University（斯坦福大学电气工程系）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；Department of Radiology, Stanford University（斯坦福大学放射学系）；Center for Artificial Intelligence in Medicine and Imaging, Stanford University（斯坦福大学医学与影像人工智能中心）
Tissue Image Analytics Centre, Department of Computer Science, University of Warwick（华威大学计算机科学系组织图像分析中心）
From the simple measurement of tissue attributes in pathology workflow to designing an explainable diagnostic/prognostic AI tool, access to accurate semantic segmentation of tissue regions in histology images is a prerequisite. However, delineating different tissue regions manually is a laborious, time-consuming and costly task that requires expert knowledge. On the other hand, the state-of-the-art automatic deep learning models for semantic segmentation require lots of annotated training data and there are only a limited number of tissue region annotated images publicly available. To obviate …
Delft University of Technology（代尔夫特理工大学）；Erasmus University Medical Center（伊拉斯姆斯大学医学中心）；Academic Medical Center（学术医学中心）
Cribriform growth patterns in prostate carcinoma are associated with poor prognosis. We aimed to introduce a deep learning method to detect such patterns automatically. To do so, convolutional neural network was trained to detect cribriform growth patterns on 128 prostate needle biopsies. Ensemble learning taking into account other tumor growth patterns during training was used to cope with heterogeneous and limited tumor tissue occurrences. ROC and FROC analyses were applied to assess network performance regarding detection of biopsies harboring cribriform growth pattern. The ROC analysis yie…
Promptable cancer segmentation using minimal expert-curated data
Department of Medical Physics and Biomedical Engineering, University College London（医学物理与生物医学工程系，伦敦大学学院）；UCL Hawkes Institute, University College London（UCL Hawkes研究所，伦敦大学学院）；Division of Surgery and Interventional Science, University College London（外科与介入科学系，伦敦大学学院）；Department of Radiology（放射科；泌尿科，斯坦福大学）；and Department of Urology, Stanford University
Mucous glands lesions analysis and assessing of malignant potential of colon polyps are very important tasks of surgical pathology. However, differential diagnosis of colon polyps often seems impossible by classical methods and it is necessary to involve computer methods capable of assessing minimal differences to extend the capabilities of the classical pathology examination. Accurate segmentation of mucous glands from histology images is a crucial step to obtain reliable morphometric criteria for quantitative diagnostic methods. We review major trends in histological images segmentation and …
University of Cambridge（剑桥大学）；Chinese University of Hong Kong（香港中文大学）
We address the problem of automated nuclear segmentation, classification, and quantification from Haematoxylin and Eosin stained histology images, which is of great relevance for several downstream computational pathology applications. In this work, we present a solution framed as a simultaneous semantic and instance segmentation framework. Our solution is part of the Colon Nuclei Identification and Counting (CoNIC) Challenge. We first train a semantic and instance segmentation model separately. Our framework uses as backbone HoverNet and Cascade Mask-RCNN models. We then ensemble the results …
University Hospital and University of Zurich（苏黎世大学医院和苏黎世大学）
Making histopathology image classifiers robust to a wide range of real-world variability is a challenging task. Here, we describe a candidate deep learning solution for the Mitosis Domain Generalization Challenge 2022 (MIDOG) to address the problem of generalization for mitosis detection in images of hematoxylin-eosin-stained histology slides under high variability (scanner, tissue type and species variability). Our approach consists in training a rotation-invariant deep learning model using aggressive data augmentation with a training set enriched with hard negative examples and automatically…
The Institute of Cancer Research（癌症研究所）；University of Warwick（华威大学）；University Hospitals Coventry and Warwickshire（考文垂与华威郡大学医院）；The Alan Turing Institute（艾伦·图灵研究所）
Object segmentation and structure localization are important steps in automated image analysis pipelines for microscopy images. We present a convolution neural network (CNN) based deep learning architecture for segmentation of objects in microscopy images. The proposed network can be used to segment cells, nuclei and glands in fluorescence microscopy and histology images after slight tuning of input parameters. The network trains at multiple resolutions of the input image, connects the intermediate layers for better localization and context and generates the output using multi-resolution decon…
National Institute of Neurological Disorders and Stroke（美国国家神经疾病与中风研究所）；University of Houston（休斯顿大学）
Reliable large-scale cell detection and segmentation is the fundamental first step to understanding biological processes in the brain. The ability to phenotype cells at scale can accelerate preclinical drug evaluation and system-level brain histology studies. The impressive advances in deep learning offer a practical solution to cell image detection and segmentation. Unfortunately, categorizing cells and delineating their boundaries for training deep networks is an expensive process that requires skilled biologists. This paper presents a novel self-supervised Dual-Loss Adaptive Masked Autoenco…
