NephroNet: A Novel Program for Identifying Renal Cell Carcinoma and Generating Synthetic Training Images with Convolutional Neural Networks and Diffusion Models
Renal cell carcinoma (RCC) is a type of cancer that originates in the kidneys and is the most common type of kidney cancer in adults. It can be classified into several subtypes, including clear cell RCC, papillary RCC, and chromophobe RCC. In this study, an artificial intelligence model was developed and trained for classifying different subtypes of RCC using ResNet-18, a convolutional neural network that has been widely used for image classification tasks. The model was trained on a dataset of RCC histopathology images, which consisted of digital images of RCC surgical resection slides that were annotated with the corresponding subtype labels. The performance of the trained model was evaluated using several metrics, including accuracy, precision, and recall. Additionally, in this research, a novel synthetic image generation tool, NephroNet, is developed on diffusion models that are used to generate original images of RCC surgical resection slides. Diffusion models are a class of generative models capable of synthesizing high-quality images from noise. Several diffusers such as Stable Diffusion, Dreambooth Text-to-Image, and Textual Inversion were trained on a dataset of RCC images and were used to generate a series of original images that resembled RCC surgical resection slides, all within the span of fewer than four seconds. The generated images were visually realistic and could be used for creating new training datasets, testing the performance of image analysis algorithms, and training medical professionals. NephroNet is provided as an open-source software package and contains files for data preprocessing, training, and visualization. Overall, this study demonstrates the potential of artificial intelligence and diffusion models for classifying and generating RCC images, respectively. These methods could be useful for improving the diagnosis and treatment of RCC and more.
Xi’an Jiaotong University（西安交通大学）；Pennsylvania State University（宾夕法尼亚州立大学）；the First Affiliated Hospital of Xi’an Jiaotong University（西安交通大学第一附属医院）
Diagnostic pathology, which is the basis and gold standard of cancer diagnosis, provides essential information on the prognosis of the disease and vital evidence for clinical treatment. Tumor region detection, subtype and grade classification are the fundamental diagnostic indicators for renal cell carcinoma (RCC) in whole-slide images (WSIs). However, pathological diagnosis is subjective, differences in observation and diagnosis between pathologists is common in hospitals with inadequate diagnostic capacity. The main challenge for developing deep learning based RCC diagnostic system is the la…
Clear cell renal cell carcinoma (ccRCC) is one of the most common forms of intratumoral heterogeneity in the study of renal cancer. ccRCC originates from the epithelial lining of proximal convoluted renal tubules. These cells undergo abnormal mutations in the presence of Ki67 protein and create a lump-like structure through cell proliferation. Manual counting of tumor cells in the tissue-affected sections is one of the strongest prognostic markers for renal cancer. However, this procedure is time-consuming and also prone to subjectivity. These assessments are based on the physical cell appeara…
Obtaining a large amount of labeled data in medical imaging is laborious and time-consuming, especially for histopathology. However, it is much easier and cheaper to get unlabeled data from whole-slide images (WSIs). Semi-supervised learning (SSL) is an effective way to utilize unlabeled data and alleviate the need for labeled data. For this reason, we proposed a framework that employs an SSL method to accurately detect cancerous regions with a novel annotation method called Minimal Point-Based annotation, and then utilize the predicted results with an innovative hybrid loss to train a classif…
Geisel School of Medicine at Dartmouth（达特茅斯盖泽尔医学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希契科克医疗中心）；Dartmouth College（达特茅斯学院）
Renal cell carcinoma (RCC) is the most common renal cancer in adults. The histopathologic classification of RCC is essential for diagnosis, prognosis, and management of patients. Reorganization and classification of complex histologic patterns of RCC on biopsy and surgical resection slides under a microscope remains a heavily specialized, error-prone, and time-consuming task for pathologists. In this study, we developed a deep neural network model that can accurately classify digitized surgical resection slides and biopsy slides into five related classes: clear cell RCC, papillary RCC, chromop…
The grade of clear cell renal cell carcinoma (ccRCC) is a critical prognostic factor, making ccRCC nuclei grading a crucial task in RCC pathology analysis. Computer-aided nuclei grading aims to improve pathologists' work efficiency while reducing their misdiagnosis rate by automatically identifying the grades of tumor nuclei within histopathological images. Such a task requires precisely segment and accurately classify the nuclei. However, most of the existing nuclei segmentation and classification methods can not handle the inter-class similarity property of nuclei grading, thus can not be di…
MeD-3D: A Multimodal Deep Learning Framework for Precise Recurrence Prediction in Clear Cell Renal Cell Carcinoma (ccRCC)
Skolkovo Institute of Science and Technology (Skoltech)（斯克罗夫诺科学与技术学院（Skoltech））；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））
Background: Clear cell renal cell carcinoma (ccRCC) is the most common renal-related tumor with high heterogeneity. There is still an urgent need for novel diagnostic and prognostic biomarkers for ccRCC. Methods: We proposed a weakly-supervised deep learning strategy using conventional histology of 1752 whole slide images from multiple centers. Our study was demonstrated through internal cross-validation and external validations for the deep learning-based models. Results: Automatic diagnosis for ccRCC through intelligent subtyping of renal cell carcinoma was proved in this study. Our graderis…
Renal cell carcinoma (RCC) is a common cancer that varies in clinical behavior. Indolent RCC is often low-grade without necrosis and can be monitored without treatment. Aggressive RCC is often high-grade and can cause metastasis and death if not promptly detected and treated. While most kidney cancers are detected on CT scans, grading is based on histology from invasive biopsy or surgery. Determining aggressiveness on CT images is clinically important as it facilitates risk stratification and treatment planning. This study aims to use machine learning methods to identify radiology features tha…
Institute of Imaging and Computer Vision, RWTH Aachen University（亚琛工业大学成像与计算机视觉研究所）；Heinrich Heine University Düsseldorf（杜塞尔多夫海因里希·海涅大学）；Institute of Medical Sciences, Kermanshah University（克尔曼沙阿大学医学研究所）
University of St Andrews（圣安德鲁斯大学）；Lothian NHS University Hospitals（洛锡安NHS大学医院）
The presence and density of specific types of immune cells are important to understand a patient's immune response to cancer. However, immunofluorescence staining required to identify T cell subtypes is expensive, time-consuming, and rarely performed in clinical settings. We present a framework to virtually stain Hoechst images (which are cheap and widespread) with both CD3 and CD8 to identify T cell subtypes in clear cell renal cell carcinoma using generative adversarial networks. Our proposed method jointly learns both staining tasks, incentivising the network to incorporate mutually benefic…
A Robust Deep Learning Method with Uncertainty Estimation for the Pathological Classification of Renal Cell Carcinoma based on CT Images
School of Computer and Communication Engineering, Zhengzhou University of Light Industry（郑州轻工业大学计算机与通信工程学院）；Sun Yat-Sen University Cancer Center（中山大学肿瘤防治中心）；Michigan Technological University（密歇根理工大学）；The First People's Hospital of Guangzhou（广州市第一人民医院）
Beyond Discrimination: Evaluating Probability Calibration in Radiomics-Image Fusion for Renal Cell Carcinoma Classification
Research Ireland Centre for Research Training in Machine Learning（爱尔兰研究机器学习研究中心）；ML-Labs（机器学习实验室）；School of Computer Science, University College Dublin（都柏林大学计算机学院）；School of Computer Science, Technological University Dublin（都柏林理工大学计算机学院）；School of Veterinary Medicine, University College Dublin（都柏林大学兽医学院）；School of Computing, Dublin City University（都柏林城市大学计算机学院）
Purpose: To identify optimal classification methods for computed tomography (CT) radiomics-based preoperative prediction of clear cells renal cell carcinoma (ccRCC) grade. Methods and material: Seventy one ccRCC patients were included in the study. Three image preprocessing techniques (Laplacian of Gaussian, wavelet filter, and discretization of the intensity values) were applied on tumor volumes. In total, 2530 radiomics features (tumor shape and size, intensity statistics, and texture) were extracted from each segmented tumor volume. Univariate analysis was performed to assess the associatio…
Unraveling Radiomics Complexity: Strategies for Optimal Simplicity in Predictive Modeling
Université de Sherbrooke（谢布鲁克大学）；OncoRay – National Center for Radiation Research in Oncology, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology, Helmholtz-Zentrum Dresden-Rossendorf（德累斯顿工业大学卡尔·古斯塔夫·卡鲁斯医学院及大学医院OncoRay肿瘤放射研究国家中心，德累斯顿-罗森多夫亥姆霍兹研究中心）；Cedars-Sinai Medical Center（西达赛奈医疗中心）；University of California San Francisco（加利福尼亚大学旧金山分校）；Stanford University（斯坦福大学）
Improve Cross-Modality Segmentation by Treating T1-Weighted MRI Images as Inverted CT Scans
Charité - Universitätsmedizin Berlin（柏林查理医院）；Freie Universität Berlin（柏林自由大学）；Humboldt Universität zu Berlin（柏林洪堡大学）；Radboud University Medical Center（拉德伯德大学医学中心）；Massachusetts General Hospital and Harvard Medical School（麻省总医院和哈佛医学院）；Technical University of Munich（慕尼黑技术大学）
Beyond Gate Weights: Decision-Level Interpretation of Radiomics-Image Feature Fusion for RCC Classification
Research Ireland Centre for Research Training in Machine Learning（爱尔兰研究机器学习研究培训中心）；School of Computer Science, University College Dublin（都柏林大学计算机学院）；School of Veterinary Medicine, University College Dublin（都柏林大学兽医学院）
