Application of deep learning in digital pathology shows promise on improving disease diagnosis and understanding. We present a deep generative model that learns to simulate high-fidelity cancer tissue images while mapping the real images onto an interpretable low dimensional latent space. The key to the model is an encoder trained by a previously developed generative adversarial network, PathologyGAN. We study the latent space using 249K images from two breast cancer cohorts. We find that the latent space encodes morphological characteristics of tissues (e.g. patterns of cancer, lymphocytes, and stromal cells). In addition, the latent space reveals distinctly enriched clusters of tissue architectures in the high-risk patient group.
Histopathological images of tumors contain abundant information about how tumors grow and how they interact with their micro-environment. Better understanding of tissue phenotypes in these images could reveal novel determinants of pathological processes underlying cancer, and in turn improve diagnosis and treatment options. Advances of Deep learning makes it ideal to achieve those goals, however, its application is limited by the cost of high quality labels from patients data. Unsupervised learning, in particular, deep generative models with representation learning properties provides an alter…
Digitization of histopathology slides has led to several advances, from easy data sharing and collaborations to the development of digital diagnostic tools. Deep learning (DL) methods for classification and detection have shown great potential, but often require large amounts of training data that are hard to collect, and annotate. For many cancer types, the scarceness of data creates barriers for training DL models. One such scenario relates to detecting tumor metastasis in lymph node tissue, where the low ratio of tumor to non-tumor cells makes the diagnostic task hard and time-consuming. DL…
Structural and mechanical differences between cancerous and healthy tissue give rise to variations in macroscopic properties such as visual appearance and elastic modulus that show promise as signatures for early cancer detection. Atomic force microscopy (AFM) has been used to measure significant differences in stiffness between cancerous and healthy cells owing to its high force sensitivity and spatial resolution, however due to absorption and scattering of light, it is often challenging to accurately locate where AFM measurements have been made on a bulk tissue sample. In this paper we descr…
University of Glasgow（格拉斯哥大学）；New York University（纽约大学）
Deep learning based analysis of histopathology images shows promise in advancing the understanding of tumor progression, tumor micro-environment, and their underpinning biological processes. So far, these approaches have focused on extracting information associated with annotations. In this work, we ask how much information can be learned from the tissue architecture itself. We present an adversarial learning model to extract feature representations of cancer tissue, without the need for manual annotations. We show that these representations are able to identify a variety of morphological char…
Kyushu University（九州大学）；Kyoto University Hospital（京都大学医院）
Pathological diagnosis is used for examining cancer in detail, and its automation is in demand. To automatically segment each cancer area, a patch-based approach is usually used since a Whole Slide Image (WSI) is huge. However, this approach loses the global information needed to distinguish between classes. In this paper, we utilized the Distance from the Boundary of tissue (DfB), which is global information that can be extracted from the original image. We experimentally applied our method to the three-class classification of cervical cancer, and found that it improved the total performance …
Microscopic examination of tissues or histopathology is one of the diagnostic procedures for detecting colorectal cancer. The pathologist involved in such an examination usually identifies tissue type based on texture analysis, especially focusing on tumour-stroma ratio. In this work, we automate the task of tissue classification within colorectal cancer histology samples using deep transfer learning. We use discriminative fine-tuning with one-cycle-policy and apply structure-preserving colour normalization to boost our results. We also provide visual explanations of the deep neural network's …
Khalifa University（哈利法大学）；Beihang University（北京航空航天大学）；University of Louisville（路易斯维尔大学）
Prostate cancer (PCa) is the second deadliest form of cancer in males, and it can be clinically graded by examining the structural representations of Gleason tissues. This paper proposes \RV{a new method} for segmenting the Gleason tissues \RV{(patch-wise) in order to grade PCa from the whole slide images (WSI).} Also, the proposed approach encompasses two main contributions: 1) A synergy of hybrid dilation factors and hierarchical decomposition of latent space representation for effective Gleason tissues extraction, and 2) A three-tiered loss function which can penalize different semantic seg…
Vietnam Aviation Academy（越南航空学院）；Ho Chi Minh City Open University（胡志明市开放大学）；University of Information Technology, VNU-HCM（胡志明市国家大学信息技术大学）
According to some medical imaging techniques, breast histopathology images called Hematoxylin and Eosin are considered as the gold standard for cancer diagnoses. Based on the idea of dividing the pathologic image (WSI) into multiple patches, we used the window [512,512] sliding from left to right and sliding from top to bottom, each sliding step overlapping by 50% to augmented data on a dataset of 400 images which were gathered from the ICIAR 2018 Grand Challenge. Then use the EffficientNet model to classify and identify the histopathological images of breast cancer into 4 types: Normal, Benig…
University of Oxford（牛津大学）；University Hospitals Coventry and Warwickshire（考文垂与沃里克郡大学医院）；University of Warwick（华威大学）；Hamad Medical Corporation（哈马德医疗公司）；International Association for Cancer Research（国际癌症研究协会）
Distant metastasis is the major cause of death in colorectal cancer (CRC). Patients at high risk of developing distant metastasis could benefit from appropriate adjuvant and follow-up treatments if stratified accurately at an early stage of the disease. Studies have increasingly recognized the role of diverse cellular components within the tumor microenvironment in the development and progression of CRC tumors. In this paper, we show that a new method of automated analysis of digitized images from colorectal cancer tissue slides can provide important estimates of distant metastasis-free surviv…
IBM Research - Europe（IBM欧洲研究院）；University College Dublin（都柏林大学学院）；Royal College of Surgeons（皇家外科医学院）
Intra-operative identification of malignant versus benign or healthy tissue is a major challenge in fluorescence guided cancer surgery. We propose a perfusion quantification method for computer-aided interpretation of subtle differences in dynamic perfusion patterns which can be used to distinguish between normal tissue and benign or malignant tumors intra-operatively in real-time by using multispectral endoscopic videos. The method exploits the fact that vasculature arising from cancer angiogenesis gives tumors differing perfusion patterns from the surrounding tissue, and defines a signature …
University of Warwick（华威大学）；Tissue Image Analytics (TIA) Centre, University of Warwick（华威大学组织图像分析中心）
The problem of recognizing various types of tissues present in multi-gigapixel histology images is an important fundamental pre-requisite for downstream analysis of the tumor microenvironment in a bottom-up analysis paradigm for computational pathology. In this paper, we propose a deep dictionary learning approach to solve the problem of tissue phenotyping in histology images. We propose deep Multi-Resolution Dictionary Learning (deepMRDL) in order to benefit from deep texture descriptors at multiple different spatial resolutions. We show the efficacy of the proposed approach through extensive…
The quantitative measurement of structural alterations at the nanoscale level is important for understanding the physical state of biological samples. Studies have shown that the progression of cancer is associated with the rearrangements of building blocks of cells/tissues such as DNA, RNA, lipids, etc. Partial wave spectroscopy is a recently developed mesoscopic physics-based spectroscopic imaging technique which can detect such nanoscale changes in cells/tissues. At present, chemotherapy drug treatment is the only effective form of treatment; however, the development of drug-resistant cance…
Automated HER2 Scoring in Breast Cancer Images Using Deep Learning and Pyramid Sampling
University of California, Los Angeles（加州大学洛杉矶分校）；California NanoSystems Institute（加州纳米系统研究所）；University of California at Davis（加州大学戴维斯分校）；Bnai-Zion Medical Center（贝内锡安医疗中心）；Hadassah Hebrew University Medical Center（哈达萨希伯来大学医疗中心）；David Geffen School of Medicine, University of California, Los Angeles（加州大学洛杉矶分校大卫·格芬医学院）
FECT: Classification of Breast Cancer Pathological Images Based on Fusion Features
Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School（清华深圳国际研究生院生物制药与健康工程研究所）；School of Medicine, Nankai University（南开大学医学院）
Hamburg University of Technology（汉堡工业大学）；University Medical Centre Schleswig-Holstein（石勒苏益格-荷尔斯泰因大学医学中心）
Purpose: The gold standard for colorectal cancer metastases detection in the peritoneum is histological evaluation of a removed tissue sample. For feedback during interventions, real-time in-vivo imaging with confocal laser microscopy has been proposed for differentiation of benign and malignant tissue by manual expert evaluation. Automatic image classification could improve the surgical workflow further by providing immediate feedback. Methods: We analyze the feasibility of classifying tissue from confocal laser microscopy in the colon and peritoneum. For this purpose, we adopt both classical…
Characterization of breast parenchyma on dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a challenging task owing to the complexity of underlying tissue structures. Current quantitative approaches, including radiomics and deep learning models, do not explicitly capture the complex and subtle parenchymal structures, such as fibroglandular tissue. In this paper, we propose a novel method to direct a neural network's attention to a dedicated set of voxels surrounding biologically relevant tissue structures. By extracting multi-dimensional topological structures with high salienc…
GRAPHITE: Graph-Based Interpretable Tissue Examination for Enhanced Explainability in Breast Cancer Histopathology
School of Computer Science and Engineering（计算机科学与工程学院）；University of New South Wales（新南威尔士大学）；St. George and Sutherland Clinical School（圣乔治与萨瑟兰临床学院）；Department of Anatomical Pathology（解剖病理学部）；NSW Health Pathology（新南威尔士州卫生病理学）；Cancer Care Centre（癌症护理中心）
Current methods for diagnosing the progression of multiple types of cancer within patients rely on interpreting stained needle biopsies. This process is time-consuming and susceptible to error throughout the paraffinization, Hematoxylin and Eosin (H&E) staining, deparaffinization, and annotation stages. Fourier Transform Infrared (FTIR) imaging has been shown to be a promising alternative to staining for appropriately annotating biopsy cores without the need for deparaffinization or H&E staining with the use of Fourier Transform Infrared (FTIR) images when combined with machine learning to int…
EPFL（洛桑联邦理工学院）；Bern（伯尔尼大学）；Lausanne University Hospital（洛桑大学医院）；Center of Biomedical Imaging（生物医学成像中心）
With the long-term rapid increase in incidences of colorectal cancer (CRC), there is an urgent clinical need to improve risk stratification. The conventional pathology report is usually limited to only a few histopathological features. However, most of the tumor microenvironments used to describe patterns of aggressive tumor behavior are ignored. In this work, we aim to learn histopathological patterns within cancerous tissue regions that can be used to improve prognostic stratification for colorectal cancer. To do so, we propose a self-supervised learning method that jointly learns a represen…
The analysis of FFPE tissue sections stained with haematoxylin and eosin (H&E) or immunohistochemistry (IHC) is an essential part of the pathologic assessment of surgically resected breast cancer specimens. IHC staining has been broadly adopted into diagnostic guidelines and routine workflows to manually assess status and scoring of several established biomarkers, including ER, PGR, HER2 and KI67. However, this is a task that can also be facilitated by computational pathology image analysis methods. The research in computational pathology has recently made numerous substantial advances, often …
Worldwide, prostate cancer is one of the main cancers affecting men. The final diagnosis of prostate cancer is based on the visual detection of Gleason patterns in prostate biopsy by pathologists. Computer-aided-diagnosis systems allow to delineate and classify the cancerous patterns in the tissue via computer-vision algorithms in order to support the physicians' task. The methodological core of this work is a U-Net convolutional neural network for image segmentation modified with residual blocks able to segment cancerous tissue according to the full Gleason system. This model outperforms othe…
Geisel School of Medicine at Dartmouth（达特茅斯盖泽尔医学院）；Dartmouth College（达特茅斯学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希契科克医疗中心）
Deep learning-based methods, such as the sliding window approach for cropped-image classification and heuristic aggregation for whole-slide inference, for analyzing histological patterns in high-resolution microscopy images have shown promising results. These approaches, however, require a laborious annotation process and are fragmented. This diagnostic study collected deidentified high-resolution histological images (N = 379) for training a new model composed of a convolutional neural network and a grid-based attention network, trainable without region-of-interest annotations. Histological im…
Deep learning classifiers for characterization of whole slide tissue morphology require large volumes of annotated data to learn variations across different tissue and cancer types. As is well known, manual generation of digital pathology training data is time consuming and expensive. In this paper, we propose a semi-automated method for annotating a group of similar instances at once, instead of collecting only per-instance manual annotations. This allows for a much larger training set, that reflects visual variability across multiple cancer types and thus training of a single network which c…
Multiplexed immuno-fluorescence tissue imaging, allowing simultaneous detection of molecular properties of cells, is an essential tool for characterizing the complex cellular mechanisms in translational research and clinical practice. New image analysis approaches are needed because tissue section stained with a mixture of protein, DNA and RNA biomarkers are introducing various complexities, including spurious edges due to fluorescent staining artifacts between touching or overlapping cells. We have developed the RRScell method harnessing the stochastic random-reaction-seed (RRS) algorithm and…
The distribution and appearance of nuclei are essential markers for the diagnosis and study of cancer. Despite the importance of nuclear morphology, there is a lack of large scale, accurate, publicly accessible nucleus segmentation data. To address this, we developed an analysis pipeline that segments nuclei in whole slide tissue images from multiple cancer types with a quality control process. We have generated nucleus segmentation results in 5,060 Whole Slide Tissue images from 10 cancer types in The Cancer Genome Atlas. One key component of our work is that we carried out a multi-level qual…
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Weill Cornell Graduate School for Medical Sciences（威尔康奈尔医学研究生院）
Pathologic analysis of surgical excision specimens for breast carcinoma is important to evaluate the completeness of surgical excision and has implications for future treatment. This analysis is performed manually by pathologists reviewing histologic slides prepared from formalin-fixed tissue. In this paper, we present Deep Multi-Magnification Network trained by partial annotation for automated multi-class tissue segmentation by a set of patches from multiple magnifications in digitized whole slide images. Our proposed architecture with multi-encoder, multi-decoder, and multi-concatenation out…
East China Normal University（华东师范大学）；School of Computer Science, East China Normal University（华东师范大学计算机科学学院）
Prostate cancer (PCa) is one of the most common cancers in men around the world. The most accurate method to evaluate lesion levels of PCa is microscopic inspection of stained biopsy tissue and estimate the Gleason score of tissue microarray (TMA) image by expert pathologists. However, it is time-consuming for pathologists to identify the cellular and glandular patterns for Gleason grading in large TMA images. We used Gleason2019 Challenge dataset to build a convolutional neural network (CNN) model to segment TMA images to regions of different Gleason grades and predict the Gleason score accor…
The ACROBAT 2022 Challenge: Automatic Registration Of Breast Cancer Tissue
Karolinska Institutet（卡罗林斯卡学院）；University of Turku（图尔库大学）；University of Eastern Finland（东芬兰大学）；Zealand University Hospital（西兰大学医院）；Gestalt Diagnostics（格式塔诊断公司）；Moffitt Cancer Center（莫菲特癌症中心）；University of Applied Sciences Western Switzerland（瑞士西部应用科学大学）；AGH University of Kraków（克拉科夫AGH科技大学）；University of Geneva（日内瓦大学）；University of Padova（帕多瓦大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）；University of Twente（特温特大学）；Muroran Institute of Technology（室兰工业大学）；Niigata University of Health and Welfare（新潟医疗福祉大学）；Konica Minolta, Inc.（柯尼卡美能达株式会社）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）；Karolinska University Hospital（卡罗林斯卡大学医院）；Tampere University（坦佩雷大学）
Cancer is the uncontrollable cell division of abnormal cells inside the human body, which can spread to other body organs. It is one of the non-communicable diseases (NCDs) and NCDs accounts for 71% of total deaths worldwide whereas lung cancer is the second most diagnosed cancer after female breast cancer. Cancer survival rate of lung cancer is only 19%. There are various methods for the diagnosis of lung cancer, such as X-ray, CT scan, PET-CT scan, bronchoscopy and biopsy. However, to know the subtype of lung cancer based on the tissue type H and E staining is widely used, where the staining…
James A. Haley Veterans’ Hospital（詹姆斯·A·黑利退伍军人医院）；University of South Florida（南佛罗里达大学）；Moffitt Cancer Center（莫菲特癌症中心）
The field of Machine Learning, a subset of Artificial Intelligence, has led to remarkable advancements in many areas, including medicine. Machine Learning algorithms require large datasets to train computer models successfully. Although there are medical image datasets available, more image datasets are needed from a variety of medical entities, especially cancer pathology. Even more scarce are ML-ready image datasets. To address this need, we created an image dataset (LC25000) with 25,000 color images in 5 classes. Each class contains 5,000 images of the following histologic entities: colon a…
A tissue and cell-level annotated H&E and PD-L1 histopathology image dataset in non-small cell lung cancer
Radboud University Medical Center（拉德堡德大学医学中心）；University of Brescia（布雷西亚大学）；Ospedale Sacro Cuore（圣十字医院）；Provincial Hospital of Bolzano (SABES-ASDAA)（博尔扎诺省医院）；University and Hospital Trust of Verona（威尼斯大学与医院信托）；Karolinska University Hospital（卡罗林斯卡大学医院）；Biopticka Laboratory, Ltd（Biopticka实验室）
Serous borderline ovarian tumor (SBOT) and high-grade serous ovarian cancer (HGSOC) are two distinct subtypes of epithelial ovarian tumors, with markedly different biologic background, behavior, prognosis, and treatment. However, the histologic diagnosis of serous ovarian tumors can be subjectively variable and labor-intensive as multiple tumor slides/blocks need to be thoroughly examined to search for these features. In this study, we aimed to evaluate technical feasibility of using digital pathological approaches to facilitate objective and scalable diagnosis screening for SBOT and HGSOC. Ba…
Magnetic resonance imaging (MRI) has great potential to improve prostate cancer diagnosis. It can spare men with a normal exam from undergoing invasive biopsy while making biopsies more accurate in men with lesions suspicious for cancer. Yet, the subtle differences between cancer and confounding conditions, render the interpretation of MRI challenging. The tissue collected from patients that undergo pre-surgical MRI and radical prostatectomy provides a unique opportunity to correlate histopathology images of the entire prostate with MRI in order to accurately map the extent of prostate cancer …
Objective: Alterations of Young's modulus (YM) and Poisson's ratio (PR) in biological tissues are often early indicators of the onset of pathological conditions. Knowledge of these parameters has been proven to be of great clinical significance for the diagnosis, prognosis and treatment of cancers. Currently, however, there are no non-invasive modalities that can be used to image and quantify these parameters in vivo without assuming incompressibility of the tissue, an assumption that is rarely justified in human tissues. Methods: In this paper, we develop a new method to simultaneously recons…
