To train a robust deep learning model, one usually needs a balanced set of categories in the training data. The data acquired in a medical domain, however, frequently contains an abundance of healthy patients, versus a small variety of positive, abnormal cases. Moreover, the annotation of a positive sample requires time consuming input from medical domain experts. This scenario would suggest a promise for one-class classification type approaches. In this work we propose a general one-class classification model for histology, that is meta-trained on multiple histology datasets simultaneously, and can be applied to new tasks without expensive re-training. This model could be easily used by pathology domain experts, and potentially be used for screening purposes.
Microscopic examination of tissues or histopathology is one of the diagnostic procedures for detecting colorectal cancer. The pathologist involved in such an examination usually identifies tissue type based on texture analysis, especially focusing on tumour-stroma ratio. In this work, we automate the task of tissue classification within colorectal cancer histology samples using deep transfer learning. We use discriminative fine-tuning with one-cycle-policy and apply structure-preserving colour normalization to boost our results. We also provide visual explanations of the deep neural network's …
Qatar University（卡塔尔大学）；Izmir University of Economics（伊兹密尔经济大学）；Tampere University of Technology（坦佩雷理工大学）
Computer-aided diagnosis (CAD) based on histopathological imaging has progressed rapidly in recent years with the rise of machine learning based methodologies. Traditional approaches consist of training a classification model using features extracted from the images, based on textures or morphological properties. Recently, deep-learning based methods have been applied directly to the raw (unprocessed) data. However, their usability is impacted by the paucity of annotated data in the biomedical sector. In order to leverage the learning capabilities of deep Convolutional Neural Nets (CNNs) withi…
University of Saskatchewan（萨斯喀彻温大学）；University of Tulane（杜兰大学）
Breast cancer is one of the most common causes of cancer-related death in women worldwide. Early and accurate diagnosis of breast cancer may significantly increase the survival rate of patients. In this study, we aim to develop a fully automatic, deep learning-based, method using descriptor features extracted by Deep Convolutional Neural Network (DCNN) models and pooling operation for the classification of hematoxylin and eosin stain (H&E) histological breast cancer images provided as a part of the International Conference on Image Analysis and Recognition (ICIAR) 2018 Grand Challenge on BreAs…
Lung cancer is the leading cause of cancer-related death worldwide. Lung adenocarcinoma (LUAD) and lung squamous cell carcinoma (LUSC) are the most common histologic subtypes of non-small-cell lung cancer (NSCLC). Histology is an essential tool for lung cancer diagnosis. Pathologists make classifications according to the dominant subtypes. Although morphology remains the standard for diagnosis, significant tool needs to be developed to elucidate the diagnosis. In our study, we utilize the pre-trained Vision Transformer (ViT) model to classify multiple label lung cancer on histologic slices (fr…
University of Ljubljana（卢布尔雅那大学）；XLAB d.o.o.（XLAB 有限责任公司）
Detection of visual anomalies refers to the problem of finding patterns in different imaging data that do not conform to the expected visual appearance and is a widely studied problem in different domains. Due to the nature of anomaly occurrences and underlying generating processes, it is hard to characterize them and obtain labeled data. Obtaining labeled data is especially difficult in biomedical applications, where only trained domain experts can provide labels, which often come in large diversity and complexity. Recently presented approaches for unsupervised detection of visual anomalies a…
This paper presents results of applying Inception v4 deep convolutional neural network to ICIAR-2018 Breast Cancer Classification Grand Challenge, part a. The Challenge task is to classify breast cancer biopsy results, presented in form of hematoxylin and eosin stained images. Breast cancer classification is of primary interest to the medical practitioners and thus binary classification of breast cancer images have been under investigation by many researchers, but multi-class categorization of histology breast images have been challenging due to the subtle differences among the categories. In …
Microsatellite instability (MSI) is a tumor phenotype whose diagnosis largely impacts patient care in colorectal cancers (CRC), and is associated with response to immunotherapy in all solid tumors. Deep learning models detecting MSI tumors directly from H&E stained slides have shown promise in improving diagnosis of MSI patients. Prior deep learning models for MSI detection have relied on neural networks pretrained on ImageNet dataset, which does not contain any medical image. In this study, we leverage recent advances in self-supervised learning by training neural networks on histology images…
Multi-Scale Hybrid Vision Transformer for Learning Gastric Histology: AI-Based Decision Support System for Gastric Cancer Treatment
KAIST（韩国科学技术院）；Chungnam National University School of Medicine（忠南大学医学院）；Chungnam National University Hospital（忠南大学医院）；Chungnam National University Sejong Hospital（忠南大学世宗医院）
本文提出基于两阶段混合 ViT 与多尺度自注意力的 AI 系统，实现胃癌病理五分类并匹配治疗指南，在多中心切片中验证可提升灵敏度并缩短筛查时间。
University of Warwick（华威大学）；National University of Sciences and Technology（国立科技大学）；University Hospitals Coventry（考文垂大学医院）；The Alan Turing Institute（艾伦·图灵研究所）
Digital histology images are amenable to the application of convolutional neural network (CNN) for analysis due to the sheer size of pixel data present in them. CNNs are generally used for representation learning from small image patches (e.g. 224x224) extracted from digital histology images due to computational and memory constraints. However, this approach does not incorporate high-resolution contextual information in histology images. We propose a novel way to incorporate larger context by a context-aware neural network based on images with a dimension of 1,792x1,792 pixels. The proposed fr…
Worldwide, prostate cancer is one of the main cancers affecting men. The final diagnosis of prostate cancer is based on the visual detection of Gleason patterns in prostate biopsy by pathologists. Computer-aided-diagnosis systems allow to delineate and classify the cancerous patterns in the tissue via computer-vision algorithms in order to support the physicians' task. The methodological core of this work is a U-Net convolutional neural network for image segmentation modified with residual blocks able to segment cancerous tissue according to the full Gleason system. This model outperforms othe…
University of Chicago Medicine（芝加哥大学医学中心）；Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院蒂施癌症研究所）；Ghost Autonomy, Inc.（Ghost Autonomy 公司）；University of Wisconsin at Madison（威斯康星大学麦迪逊分校）；University of Chicago（芝加哥大学）
Artificial intelligence methods including deep neural networks (DNN) can provide rapid molecular classification of tumors from routine histology with accuracy that matches or exceeds human pathologists. Discerning how neural networks make their predictions remains a significant challenge, but explainability tools help provide insights into what models have learned when corresponding histologic features are poorly defined. Here, we present a method for improving explainability of DNN models using synthetic histology generated by a conditional generative adversarial network (cGAN). We show that …
University of Warwick（华威大学）；University Hospitals Coventry and Warwickshire（考文垂和沃里克大学医院）；The Alan Turing Institute（艾伦·图灵研究所）
Synthetic images can be used for the development and evaluation of deep learning algorithms in the context of limited availability of data. In the field of computational pathology, where histology images are large in size and visual context is crucial, synthesis of large high resolution images via generative modeling is a challenging task. This is due to memory and computational constraints hindering the generation of large images. To address this challenge, we propose a novel SAFRON (Stitching Across the FRONtiers) framework to construct realistic, large high resolution tissue image tiles fro…
University of Warwick（华威大学）；Tissue Image Analytics (TIA) Centre, University of Warwick（华威大学组织图像分析中心）
The problem of recognizing various types of tissues present in multi-gigapixel histology images is an important fundamental pre-requisite for downstream analysis of the tumor microenvironment in a bottom-up analysis paradigm for computational pathology. In this paper, we propose a deep dictionary learning approach to solve the problem of tissue phenotyping in histology images. We propose deep Multi-Resolution Dictionary Learning (deepMRDL) in order to benefit from deep texture descriptors at multiple different spatial resolutions. We show the efficacy of the proposed approach through extensive…
Prostate cancer is one of the most prevalent cancers worldwide. One of the key factors in reducing its mortality is based on early detection. The computer-aided diagnosis systems for this task are based on the glandular structural analysis in histology images. Hence, accurate gland detection and segmentation is crucial for a successful prediction. The methodological basis of this work is a prostate gland segmentation based on U-Net convolutional neural network architectures modified with residual and multi-resolution blocks, trained using data augmentation techniques. The residual configuratio…
The Chinese University of Hong Kong（香港中文大学）；University of Warwick（华威大学）
Colorectal cancer (CRC) grading is typically carried out by assessing the degree of gland formation within histology images. To do this, it is important to consider the overall tissue micro-environment by assessing the cell-level information along with the morphology of the gland. However, current automated methods for CRC grading typically utilise small image patches and therefore fail to incorporate the entire tissue micro-architecture for grading purposes. To overcome the challenges of CRC grading, we present a novel cell-graph convolutional neural network (CGC-Net) that converts each large…
Cancer is the uncontrollable cell division of abnormal cells inside the human body, which can spread to other body organs. It is one of the non-communicable diseases (NCDs) and NCDs accounts for 71% of total deaths worldwide whereas lung cancer is the second most diagnosed cancer after female breast cancer. Cancer survival rate of lung cancer is only 19%. There are various methods for the diagnosis of lung cancer, such as X-ray, CT scan, PET-CT scan, bronchoscopy and biopsy. However, to know the subtype of lung cancer based on the tissue type H and E staining is widely used, where the staining…
Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Freie Universität Berlin（柏林自由大学）；Technische Hochschule Ingolstadt（英戈尔施塔特工业大学）
Due to morphological similarities, the differentiation of histologic sections of cutaneous tumors into individual subtypes can be challenging. Recently, deep learning-based approaches have proven their potential for supporting pathologists in this regard. However, many of these supervised algorithms require a large amount of annotated data for robust development. We present a publicly available dataset of 350 whole slide images of seven different canine cutaneous tumors complemented by 12,424 polygon annotations for 13 histologic classes, including seven cutaneous tumor subtypes. In inter-rate…
Advances in imaging and early cancer detection have increased interest in magnetic resonance (MR) guided focused ultrasound (MRgFUS) technologies for cancer treatment. MRgFUS ablation treatments could reduce surgical risks, preserve organ tissue/function, and improve patient quality of life. However, surgical resection and histological analysis remain the gold standard to assess cancer treatment response. For non-invasive ablation therapies such as MRgFUS, the treatment response must be determined through MR imaging biomarkers. However, current MR biomarkers are inconclusive and have not been …
Disentangled Multi-modal Learning of Histology and Transcriptomics for Cancer Characterization
Department of Clinical Neurosciences, University of Cambridge, UK（剑桥大学临床神经科学系）；Department of Health Technology & Informatics, The Hong Kong Polytechnic University（香港理工大学健康科技与信息学系）；Department of Statistics and Actuarial Science, The University of Hong Kong（香港大学统计与精算科学系）；Department of Clinical Neurosciences and Department of Applied Mathematics and Theoretical Physics, University of Cambridge（剑桥大学临床神经科学系和应用数学与理论物理系；邓迪大学科学与工程学院和医学院）；School of Science and Engineering and School of Medicine, University of Dundee, UK
National University of Sciences and Technology（国家科技大学）
Instance segmentation of nuclei and glands in the histology images is an important step in computational pathology workflow for cancer diagnosis, treatment planning and survival analysis. With the advent of modern hardware, the recent availability of large-scale quality public datasets and the community organized grand challenges have seen a surge in automated methods focusing on domain specific challenges, which is pivotal for technology advancements and clinical translation. In this survey, 126 papers illustrating the AI based methods for nuclei and glands instance segmentation published in …
Prostate cancer is one of the main diseases affecting men worldwide. The Gleason scoring system is the primary diagnostic tool for prostate cancer. This is obtained via the visual analysis of cancerous patterns in prostate biopsies performed by expert pathologists, and the aggregation of the main Gleason grades in a combined score. Computer-aided diagnosis systems allow to reduce the workload of pathologists and increase the objectivity. Recently, efforts have been made in the literature to develop algorithms aiming the direct estimation of the global Gleason score at biopsy/core level with gl…
ÉTS Montreal（蒙特利尔高等技术学院）；Goodman Cancer Research Centre（古德曼癌症研究中心）；McGill University（麦吉尔大学）
Using only global image-class labels, weakly-supervised learning methods, such as class activation mapping, allow training CNNs to jointly classify an image, and locate regions of interest associated with the predicted class. However, without any guidance at the pixel level, such methods may yield inaccurate regions. This problem is known to be more challenging with histology images than with natural ones, since objects are less salient, structures have more variations, and foreground and background regions have stronger similarities. Therefore, computer vision methods for visual interpretatio…
Fraunhofer Institute for Integrated Circuits IIS（弗劳恩霍夫集成电路研究所IIS）；Friedrich-Alexander-University, Erlangen-Nuremberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；University Hospital Erlangen, FAU Erlangen-Nuremberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学埃尔朗根大学医院）
Automatic outlining of different tissue types in digitized histological specimen provides a basis for follow-up analyses and can potentially guide subsequent medical decisions. The immense size of whole-slide-images (WSI), however, poses a challenge in terms of computation time. In this regard, the analysis of non-overlapping patches outperforms pixelwise segmentation approaches, but still leaves room for optimization. Furthermore, the division into patches, regardless of the biological structures they contain, is a drawback due to the loss of local dependencies. We propose to subdivide the WS…
Identification and quantification of nuclei in colorectal cancer haematoxylin \& eosin (H\&E) stained histology images is crucial to prognosis and patient management. In computational pathology these tasks are referred to as nuclear segmentation, classification and composition and are used to extract meaningful interpretable cytological and architectural features for downstream analysis. The CoNIC challenge poses the task of automated nuclei segmentation, classification and composition into six different types of nuclei from the largest publicly known nuclei dataset - Lizard. In this regard, w…
National Cancer Institute – IRCCS – Fondazione Pascale（国家癌症研究所——IRCCS——帕斯卡尔基金会）；IBM Research – Zurich（IBM研究院——苏黎世）；ETH Zurich（苏黎世联邦理工学院）；Institute for High Performance Computing and Networking of the Research Council of Italy, ICAR-CNR（意大利研究理事会高性能计算与网络研究所，ICAR-CNR）；University of Naples Federico II（那不勒斯费德里科二世大学）；EPFL Lausanne（洛桑联邦理工学院）
Breast cancer is the most commonly diagnosed cancer and registers the highest number of deaths for women with cancer. Recent advancements in diagnostic activities combined with large-scale screening policies have significantly lowered the mortality rates for breast cancer patients. However, the manual inspection of tissue slides by the pathologists is cumbersome, time-consuming, and is subject to significant inter- and intra-observer variability. Recently, the advent of whole-slide scanning systems have empowered the rapid digitization of pathology slides, and enabled to develop digital workfl…
Canon Medical Research Europe（佳能医疗研究欧洲公司）；Heriot-Watt University（赫瑞-瓦特大学）；University of Edinburgh（爱丁堡大学）
Deep learning shows great potential for the domain of digital pathology. An automated digital pathology system could serve as a second reader, perform initial triage in large screening studies, or assist in reporting. However, it is expensive to exhaustively annotate large histology image databases, since medical specialists are a scarce resource. In this paper, we apply the semi-supervised teacher-student knowledge distillation technique proposed by Yalniz et al. (2019) to the task of quantifying prognostic features in colorectal cancer. We obtain accuracy improvements through extending this …
The past years have seen a considerable increase in cancer cases. However, a cancer diagnosis is often complex and depends on the types of images provided for analysis. It requires highly skilled practitioners but is often time-consuming and error-prone. If Machine Learning and deep learning algorithms have been widely used, a comprehensive review of the techniques used from the pre-processing steps to the final prediction is lacking. With this review, we aim to provide a comprehensive overview of the current steps required in building efficient and accurate machine learning algorithm for canc…
University of Edinburgh（爱丁堡大学）；Heriot-Watt University（赫瑞-瓦特大学）
Autofluorescence lifetime images reveal unique characteristics of endogenous fluorescence in biological samples. Comprehensive understanding and clinical diagnosis rely on co-registration with the gold standard, histology images, which is extremely challenging due to the difference of both images. Here, we show an unsupervised image-to-image translation network that significantly improves the success of the co-registration using a conventional optimisation-based regression network, applicable to autofluorescence lifetime images at different emission wavelengths. A preliminary blind comparison …
Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Broad Institute of Harvard and MIT（哈佛-麻省理工学院博德研究所）
Cancer prognostication is a challenging task in computational pathology that requires context-aware representations of histology features to adequately infer patient survival. Despite the advancements made in weakly-supervised deep learning, many approaches are not context-aware and are unable to model important morphological feature interactions between cell identities and tissue types that are prognostic for patient survival. In this work, we present Patch-GCN, a context-aware, spatially-resolved patch-based graph convolutional network that hierarchically aggregates instance-level histology …
Promptable cancer segmentation using minimal expert-curated data
Department of Medical Physics and Biomedical Engineering, University College London（医学物理与生物医学工程系，伦敦大学学院）；UCL Hawkes Institute, University College London（UCL Hawkes研究所，伦敦大学学院）；Division of Surgery and Interventional Science, University College London（外科与介入科学系，伦敦大学学院）；Department of Radiology（放射科；泌尿科，斯坦福大学）；and Department of Urology, Stanford University
Virtual reality (VR) enables data visualization in an immersive and engaging manner, and it can be used for creating ways to explore scientific data. Here, we use VR for visualization of 3D histology data, creating a novel interface for digital pathology. Our contribution includes 3D modeling of a whole organ and embedded objects of interest, fusing the models with associated quantitative features and full resolution serial section patches, and implementing the virtual reality application. Our VR application is multi-scale in nature, covering two object levels representing different ranges of …
Clear cell renal cell carcinoma (ccRCC) is one of the most common forms of intratumoral heterogeneity in the study of renal cancer. ccRCC originates from the epithelial lining of proximal convoluted renal tubules. These cells undergo abnormal mutations in the presence of Ki67 protein and create a lump-like structure through cell proliferation. Manual counting of tumor cells in the tissue-affected sections is one of the strongest prognostic markers for renal cancer. However, this procedure is time-consuming and also prone to subjectivity. These assessments are based on the physical cell appeara…
Indian Institute of Technology, Bombay（印度理工学院孟买分校）
With the increase in the use of deep learning for computer-aided diagnosis in medical images, the criticism of the black-box nature of the deep learning models is also on the rise. The medical community needs interpretable models for both due diligence and advancing the understanding of disease and treatment mechanisms. In histology, in particular, while there is rich detail available at the cellular level and that of spatial relationships between cells, it is difficult to modify convolutional neural networks to point out the relevant visual features. We adopt an approach to model histology ti…
Histological images are critical in the diagnosis and treatment of cancers. Unfortunately, the current method for capturing these microscopy images require resource intensive tissue preparation that delays diagnosis for many days to a few weeks. To streamline this process, clinicians are limited to assessing small macroscopically representative subsets of tissues. Here, we present a combined photoacoustic remote sensing (PARS) microscope and swept source optical coherence tomography (SS-OCT) system designed to circumvent these diagnostic limitations. The proposed multimodal microscope provides…
Histopathological image segmentation is a challenging and important topic in medical imaging with tremendous potential impact in clinical practice. State of the art methods rely on hand-crafted annotations which hinder clinical translation since histology suffers from significant variations between cancer phenotypes. In this paper, we propose a weakly supervised framework for whole slide imaging segmentation that relies on standard clinical annotations, available in most medical systems. In particular, we exploit a multiple instance learning scheme for training models. The proposed framework h…
Histology method is vital in the diagnosis and prognosis of cancers and many other diseases. For the analysis of histopathological images, we need to detect and segment all gland structures. These images are very challenging, and the task of segmentation is even challenging for specialists. Segmentation of glands determines the grade of cancer such as colon, breast, and prostate. Given that deep neural networks have achieved high performance in medical images, we propose a method based on the LinkNet network for gland segmentation. We found the effects of using different loss functions. By usi…
University of Saskatchewan（萨斯喀彻温大学）；University of Tulane（杜兰大学）
Breast cancer is one of the leading causes of death across the world in women. Early diagnosis of this type of cancer is critical for treatment and patient care. Computer-aided detection (CAD) systems using convolutional neural networks (CNN) could assist in the classification of abnormalities. In this study, we proposed an ensemble deep learning-based approach for automatic binary classification of breast histology images. The proposed ensemble model adapts three pre-trained CNNs, namely VGG19, MobileNet, and DenseNet. The ensemble model is used for the feature representation and extraction s…
Label- and slide-free tissue histology using 3D epi-mode quantitative phase imaging and virtual H&E staining
University of California, Davis（加利福尼亚大学戴维斯分校）；Georgia Institute of Technology（佐治亚理工学院）；Emory University（埃默里大学）；Emory University School of Medicine（埃默里大学医学院）；UC Davis Health（加州大学戴维斯分校医疗系统）
University of Freiburg（弗莱堡大学）；Medical Center – University of Freiburg（弗莱堡大学医学中心）；Faculty of Medicine, University of Freiburg（弗莱堡大学医学院）；German Cancer Consortium (DKTK)（德国癌症联盟）；Medical School of Nanjing University（南京大学医学院）；Affiliated Drum Tower Hospital（鼓楼医院）；Hannover Medical School（汉诺威医学院）；Institute for Surgical Pathology, Medical Center – University of Freiburg（弗莱堡大学医学中心外科病理研究所）
Accurate delineation of the intraprostatic gross tumour volume (GTV) is a prerequisite for treatment approaches in patients with primary prostate cancer (PCa). Prostate-specific membrane antigen positron emission tomography (PSMA-PET) may outperform MRI in GTV detection. However, visual GTV delineation underlies interobserver heterogeneity and is time consuming. The aim of this study was to develop a convolutional neural network (CNN) for automated segmentation of intraprostatic tumour (GTV-CNN) in PSMA-PET. Methods: The CNN (3D U-Net) was trained on [68Ga]PSMA-PET images of 152 patients from …
Brigham and Women's Hospital（布莱根和妇女医院）；Harvard Medical School（哈佛医学院）；Johns Hopkins School of Medicine（约翰斯·霍普金斯大学医学院）
Histology-based grade classification is clinically important for many cancer types in stratifying patients distinct treatment groups. In prostate cancer, the Gleason score is a grading system used to measure the aggressiveness of prostate cancer from the spatial organization of cells and the distribution of glands. However, the subjective interpretation of Gleason score often suffers from large interobserver and intraobserver variability. Previous work in deep learning-based objective Gleason grading requires manual pixel-level annotation. In this work, we propose a weakly-supervised approach …
Charles Sturt University（查尔斯特大学）；University of Western Sydney（西悉尼大学）；Kent Institute Australia（肯特学院）；Asia Pacific International College（亚太国际学院）；Australian Institute of Higher Education（澳大利亚高等教育学院）；Al Iraqia University（伊拉克大学）；University of Kurdistan Hewler（库尔德斯坦赫勒大学）；AL Nisour University College（阿尔尼苏尔大学学院）
Background and Aim: Recently, deep learning using convolutional neural network has been used successfully to classify the images of breast cells accurately. However, the accuracy of manual classification of those histopathological images is comparatively low. This research aims to increase the accuracy of the classification of breast cancer images by utilizing a Patch-Based Classifier (PBC) along with deep learning architecture. Methodology: The proposed system consists of a Deep Convolutional Neural Network (DCNN) that helps in enhancing and increasing the accuracy of the classification proce…
Magnetic resonance imaging (MRI) has great potential to improve prostate cancer diagnosis. It can spare men with a normal exam from undergoing invasive biopsy while making biopsies more accurate in men with lesions suspicious for cancer. Yet, the subtle differences between cancer and confounding conditions, render the interpretation of MRI challenging. The tissue collected from patients that undergo pre-surgical MRI and radical prostatectomy provides a unique opportunity to correlate histopathology images of the entire prostate with MRI in order to accurately map the extent of prostate cancer …
Australian Breast Cancer Tissue Bank, University of Sydney（澳大利亚乳腺癌症组织库，悉尼大学）；BioMed Central Ltd（BioMed Central有限公司）
Virtual microscopy includes digitisation of histology slides and the use of computer technologies for complex investigation of diseases such as cancer. However, automated image analysis, or website publishing of such digital images, is hampered by their large file sizes. We have developed two Java based open source tools: Snapshot Creator and NDPI-Splitter. Snapshot Creator converts a portion of a large digital slide into a desired quality JPEG image. The image is linked to the patients clinical and treatment information in a customised open source cancer data management software (Caisis) in u…
