Segmentation from renal pathological images is a key step in automatic analyzing the renal histological characteristics. However, the performance of models varies significantly in different types of stained datasets due to the appearance variations. In this paper, we design a robust and flexible model for cross-stained segmentation. It is a novel multi-level deep adversarial network architecture that consists of three sub-networks: (i) a segmentation network; (ii) a pair of multi-level mirrored discriminators for guiding the segmentation network to extract domain-invariant features; (iii) a shape discriminator that is utilized to further identify the output of the segmentation network and the ground truth. Experimental results on glomeruli segmentation from renal biopsy images indicate that our network is able to improve segmentation performance on target type of stained images and use unlabeled data to achieve similar accuracy to labeled data. In addition, this method can be easily applied to other tasks.
Geisel School of Medicine at Dartmouth（达特茅斯盖泽尔医学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希契科克医疗中心）；Dartmouth College（达特茅斯学院）
Renal cell carcinoma (RCC) is the most common renal cancer in adults. The histopathologic classification of RCC is essential for diagnosis, prognosis, and management of patients. Reorganization and classification of complex histologic patterns of RCC on biopsy and surgical resection slides under a microscope remains a heavily specialized, error-prone, and time-consuming task for pathologists. In this study, we developed a deep neural network model that can accurately classify digitized surgical resection slides and biopsy slides into five related classes: clear cell RCC, papillary RCC, chromop…
Massachusetts Institute of Technology（麻省理工学院）；Harvard Medical School（哈佛医学院）；Boston University School of Medicine（波士顿大学医学院）；VA Boston Healthcare（波士顿退伍军人医疗中心）
Histopathological diagnoses of tumors in tissue biopsy after Hematoxylin and Eosin (H&E) staining is the gold standard for oncology care. H&E staining is slow and uses dyes, reagents and precious tissue samples that cannot be reused. Thousands of native nonstained RGB Whole Slide Image (RWSI) patches of prostate core tissue biopsies were registered with their H&E stained versions. Conditional Generative Adversarial Neural Networks (cGANs) that automate conversion of native nonstained RWSI to computational H&E stained images were then trained. High similarities between computational and H&E dye…
StainBridge: Stain-Aware Pairwise Registration of Serial Renal Biopsy Whole-Slide Images Across Structural and Immunohistochemical Stains
University of California, Los Angeles（加州大学洛杉矶分校）；Massachusetts General Brigham（麻省总医院布里格姆）；Vanderbilt University（范德堡大学）；University of Regensburg（雷根斯堡大学）；Weill Medical College of Cornell University（康奈尔大学韦尔医学学院）；Cornell Tech（康奈尔科技）；Northwell Health（北岸医疗）
HER2 and FISH Status Prediction in Breast Biopsy H&E-Stained Images Using Deep Learning
Indian Institute of Technology, Bombay（印度理工学院孟买分校）；Indian Council of Medical Research（印度医学研究理事会）；Tata Memorial Centre-ACTREC, HBNI（塔塔纪念中心-先进癌症治疗与研究中心，高等教育与研究国家学院）
University of Virginia（弗吉尼亚大学）；Aga Khan University（阿迦汗大学）；University of Zambia School of Medicine（赞比亚大学医学院）；Queen Mary University of London（伦敦玛丽女王大学）
Hematoxylin and Eosin (H&E) stained Whole Slide Images (WSIs) are utilized for biopsy visualization-based diagnostic and prognostic assessment of diseases. Variation in the H&E staining process across different lab sites can lead to significant variations in biopsy image appearance. These variations introduce an undesirable bias when the slides are examined by pathologists or used for training deep learning models. To reduce this bias, slides need to be translated to a common domain of stain appearance before analysis. We propose a Self-Attentive Adversarial Stain Normalization (SAASN) approac…
Current methods for diagnosing the progression of multiple types of cancer within patients rely on interpreting stained needle biopsies. This process is time-consuming and susceptible to error throughout the paraffinization, Hematoxylin and Eosin (H&E) staining, deparaffinization, and annotation stages. Fourier Transform Infrared (FTIR) imaging has been shown to be a promising alternative to staining for appropriately annotating biopsy cores without the need for deparaffinization or H&E staining with the use of Fourier Transform Infrared (FTIR) images when combined with machine learning to int…
University at Buffalo – The State University of New York（布法罗大学–纽约州立大学）；University of California, Davis Medical Center（加州大学戴维斯医学中心）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；Washington University School of Medicine（华盛顿大学医学院）；Vanderbilt University（范德堡大学）
Glomerulosclerosis, interstitial fibrosis, and tubular atrophy (IFTA) are histologic indicators of irrecoverable kidney injury. In standard clinical practice, the renal pathologist visually assesses, under the microscope, the percentage of sclerotic glomeruli and the percentage of renal cortical involvement by IFTA. Estimation of IFTA is a subjective process due to a varied spectrum and definition of morphological manifestations. Modern artificial intelligence and computer vision algorithms have the ability to reduce inter-observer variability through rigorous quantitation. In this work, we ap…
New York University（纽约大学）；IIIT Hyderabad（印度海得拉巴国际信息技术研究所）
Systemic lupus erythematosus (SLE) is an autoimmune disease in which the immune system of the patient starts attacking healthy tissues of the body. Lupus Nephritis (LN) refers to the inflammation of kidney tissues resulting in renal failure due to these attacks. The International Society of Nephrology/Renal Pathology Society (ISN/RPS) has released a classification system based on various patterns observed during renal injury in SLE. Traditional methods require meticulous pathological assessment of the renal biopsy and are time-consuming. Recently, computational techniques have helped to allevi…
Clear cell renal cell carcinoma (ccRCC) is one of the most common forms of intratumoral heterogeneity in the study of renal cancer. ccRCC originates from the epithelial lining of proximal convoluted renal tubules. These cells undergo abnormal mutations in the presence of Ki67 protein and create a lump-like structure through cell proliferation. Manual counting of tumor cells in the tissue-affected sections is one of the strongest prognostic markers for renal cancer. However, this procedure is time-consuming and also prone to subjectivity. These assessments are based on the physical cell appeara…
Renal cell carcinoma (RCC) is a common cancer that varies in clinical behavior. Indolent RCC is often low-grade without necrosis and can be monitored without treatment. Aggressive RCC is often high-grade and can cause metastasis and death if not promptly detected and treated. While most kidney cancers are detected on CT scans, grading is based on histology from invasive biopsy or surgery. Determining aggressiveness on CT images is clinically important as it facilitates risk stratification and treatment planning. This study aims to use machine learning methods to identify radiology features tha…
An Accelerated Pipeline for Multi-label Renal Pathology Image Segmentation at the Whole Slide Image Level
Vanderbilt University（范德堡大学）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；Vanderbilt University Medical Center（范德堡大学医学中心）；Georgia State University（佐治亚州立大学）
Predicting Neoadjuvant Chemotherapy Response in Triple-Negative Breast Cancer Using Pre-Treatment Histopathologic Images
The Ohio State University（俄亥俄州立大学）；University of Rochester Medical Center（罗切斯特大学医学中心）；Warren Alpert Medical School of Brown University（布朗大学沃伦·阿尔珀特医学院）；Lifespan Academic Medical Center（Lifespan学术医学中心）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
University of California, Los Angeles（加州大学洛杉矶分校）；California NanoSystems Institute (CNSI)（加州纳米系统研究所）；David Geffen School of Medicine（戴维·格芬医学院）
Pathological diagnosis relies on the visual inspection of histologically stained thin tissue specimens, where different types of stains are applied to bring contrast to and highlight various desired histological features. However, the destructive histochemical staining procedures are usually irreversible, making it very difficult to obtain multiple stains on the same tissue section. Here, we demonstrate a virtual stain transfer framework via a cascaded deep neural network (C-DNN) to digitally transform hematoxylin and eosin (H&E) stained tissue images into other types of histological stains. U…
This paper presents results of applying Inception v4 deep convolutional neural network to ICIAR-2018 Breast Cancer Classification Grand Challenge, part a. The Challenge task is to classify breast cancer biopsy results, presented in form of hematoxylin and eosin stained images. Breast cancer classification is of primary interest to the medical practitioners and thus binary classification of breast cancer images have been under investigation by many researchers, but multi-class categorization of histology breast images have been challenging due to the subtle differences among the categories. In …
East China Normal University（华东师范大学）；School of Computer Science, East China Normal University（华东师范大学计算机科学学院）
Prostate cancer (PCa) is one of the most common cancers in men around the world. The most accurate method to evaluate lesion levels of PCa is microscopic inspection of stained biopsy tissue and estimate the Gleason score of tissue microarray (TMA) image by expert pathologists. However, it is time-consuming for pathologists to identify the cellular and glandular patterns for Gleason grading in large TMA images. We used Gleason2019 Challenge dataset to build a convolutional neural network (CNN) model to segment TMA images to regions of different Gleason grades and predict the Gleason score accor…
Technion（以色列理工学院）；Technion – Israel Institute of Technology（以色列理工学院）；Cincinnati Children's Hospital Medical Center（辛辛那提儿童医院医疗中心）；University of Cincinnati College of Medicine（辛辛那提大学医学院）；Northwestern University（西北大学）；The Feinberg School of Medicine（范伯格医学院）；Ann & Robert H. Lurie Children's Hospital of Chicago（安与罗伯特·H·卢里芝加哥儿童医院）；Northwestern University Feinberg School of Medicine（西北大学范伯格医学院）
Background. Eosinophilic esophagitis (EoE) is an allergic inflammatory condition of the esophagus associated with elevated numbers of eosinophils. Disease diagnosis and monitoring requires determining the concentration of eosinophils in esophageal biopsies, a time-consuming, tedious and somewhat subjective task currently performed by pathologists. Methods. Herein, we aimed to use machine learning to identify, quantitate and diagnose EoE. We labeled more than 100M pixels of 4345 images obtained by scanning whole slides of H&E-stained sections of esophageal biopsies derived from 23 EoE patients.…
Information about the structure and composition of biopsy specimens can assist in disease monitoring and diagnosis. In principle, this can be acquired from Raman and infrared (IR) hyperspectral images (HSIs) that encode information about how a sample's constituent molecules are arranged in space. Each tissue section/component is defined by a unique combination of spatial and spectral features, but given the high dimensionality of HSI datasets, extracting and utilising them to segment images is non-trivial. Here, we show how networks based on deep convolutional autoencoders (CAEs) can perform t…
