Current analysis of tumor proliferation, the most salient prognostic biomarker for invasive breast cancer, is limited to subjective mitosis counting by pathologists in localized regions of tissue images. This study presents the first data-driven integrative approach to characterize the severity of tumor growth and spread on a categorical and molecular level, utilizing multiple biologically salient deep learning classifiers to develop a comprehensive prognostic model. Our approach achieves pathologist-level performance on three-class categorical tumor severity prediction. It additionally pioneers prediction of molecular expression data from a tissue image, obtaining a Spearman's rank correlation coefficient of 0.60 with ex vivo mean calculated RNA expression. Furthermore, our framework is applied to identify over two hundred unprecedented biomarkers critical to the accurate assessment of tumor proliferation, validating our proposed integrative pipeline as the first to holistically and objectively analyze histopathological images.
Throughout the world, breast cancer is one of the leading causes of female death. Recently, deep learning methods are developed to automatically grade breast cancer of histological slides. However, the performance of existing deep learning models is limited due to the lack of large annotated biomedical datasets. One promising way to relieve the annotating burden is to leverage the unannotated datasets to enhance the trained model. In this paper, we first apply active learning method in breast cancer grading, and propose a semi-supervised framework based on expectation maximization (EM) model. …
Higher School of Applied Sciences of Tlemcen（特莱姆森应用科学高等学院）；USTHB University（USTHB大学）；University of Mons（蒙斯大学）；University Hospital Center of Tlemcen（特莱姆森大学医院中心）；Tlemcen University（特莱姆森大学）
Computer aided detection and diagnosis systems based on deep learning have shown promising performance in breast cancer detection. However, there are cases where the obtained results lack justification. In this study, our objective is to highlight the regions of interest used by a convolutional neural network (CNN) for classifying histological images as benign or malignant. We compare these regions with the regions identified by pathologists. To achieve this, we employed the VGG19 architecture and tested three visualization methods: Gradient, LRP Z, and LRP Epsilon. Additionally, we experiment…
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）；Techno India University（印度科技大学）
Histopathological images (HI) encrypt resolution dependent heterogeneous textures & diverse color distribution variability, manifesting in micro-structural surface tissue convolutions. Also, inherently high coherency of cancerous cells poses significant challenges to breast cancer (BC) multi-classification. As such, multi-class stratification is sparsely explored & prior work mainly focus on benign & malignant tissue characterization only, which forestalls further quantitative analysis of subordinate classes like adenosis, mucinous carcinoma & fibroadenoma etc, for diagnostic competence. In th…
We present a unified framework to predict tumor proliferation scores from breast histopathology whole slide images. Our system offers a fully automated solution to predicting both a molecular data-based, and a mitosis counting-based tumor proliferation score. The framework integrates three modules, each fine-tuned to maximize the overall performance: An image processing component for handling whole slide images, a deep learning based mitosis detection network, and a proliferation scores prediction module. We have achieved 0.567 quadratic weighted Cohen's kappa in mitosis counting-based score p…
Predicting TNM stage is the major determinant of breast cancer prognosis and treatment. The essential part of TNM stage classification is whether the cancer has metastasized to the regional lymph nodes (N-stage). Pathologic N-stage (pN-stage) is commonly performed by pathologists detecting metastasis in histological slides. However, this diagnostic procedure is prone to misinterpretation and would normally require extensive time by pathologists because of the sheer volume of data that needs a thorough review. Automated detection of lymph node metastasis and pN-stage prediction has a great pote…
Technische Universität München（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Digitized Histological diagnosis is in increasing demand. However, color variations due to various factors are imposing obstacles to the diagnosis process. The problem of stain color variations is a well-defined problem with many proposed solutions. Most of these solutions are highly dependent on a reference template slide. We propose a deep-learning solution inspired by CycleGANs that is trained end-to-end, eliminating the need for an expert to pick a representative reference slide. Our approach showed superior results quantitatively and qualitatively against the state of the art methods (10%…
Eindhoven University of Technology（埃因霍温理工大学）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Harvard Medical School（哈佛医学院）；University Medical Center Utrecht（乌得勒支大学医学中心）；Radboud University Medical Center（拉德堡德大学医学中心）；Stanford University School of Medicine（斯坦福大学医学院）；University of Heidelberg（海德堡大学）；BIOQUANT；IPMB；DKFZ（德国癌症研究中心）；The Harker School（哈克学校）；ContextVision AB（ContextVision公司）；IBM Research Zürich（IBM苏黎世研究院）；IBM Research Brazil（IBM巴西研究院）；United Institute of Informatics（联合信息学研究所）；University of South Florida（南佛罗里达大学）；University of Warwick（华威大学）；Sectra（Sectra公司）；Lunit Inc.（Lunit公司）；Tsinghua University（清华大学）；Microsoft Research（微软研究院）；Beihang University（北京航空航天大学）
Tumor proliferation is an important biomarker indicative of the prognosis of breast cancer patients. Assessment of tumor proliferation in a clinical setting is highly subjective and labor-intensive task. Previous efforts to automate tumor proliferation assessment by image analysis only focused on mitosis detection in predefined tumor regions. However, in a real-world scenario, automatic mitosis detection should be performed in whole-slide images (WSIs) and an automatic method should be able to produce a tumor proliferation score given a WSI as input. To address this, we organized the TUmor Pro…
Experts' cognition-driven ensemble deep learning for external validation of predicting pathological complete response to neoadjuvant chemotherapy from histological images in breast cancer
Sichuan University（四川大学）；West China Hospital（华西医院）；Zhongjiu Flash Medical Technology Co., Ltd.（中久闪医医疗科技有限公司）；Shanxi Provincial Cancer Hospital（山西省肿瘤医院）；Sichuan Provincial People’s Hospital（四川省人民医院）；The Affiliated Hospital of Southwest Medical University（西南医科大学附属医院）
We propose a patch sampling strategy based on a sequential Monte-Carlo method for high resolution image classification in the context of Multiple Instance Learning. When compared with grid sampling and uniform sampling techniques, it achieves higher generalization performance. We validate the strategy on two artificial datasets and two histological datasets for breast cancer and sun exposure classification.
The analysis of FFPE tissue sections stained with haematoxylin and eosin (H&E) or immunohistochemistry (IHC) is an essential part of the pathologic assessment of surgically resected breast cancer specimens. IHC staining has been broadly adopted into diagnostic guidelines and routine workflows to manually assess status and scoring of several established biomarkers, including ER, PGR, HER2 and KI67. However, this is a task that can also be facilitated by computational pathology image analysis methods. The research in computational pathology has recently made numerous substantial advances, often …
Leveraging Medical Foundation Model Features in Graph Neural Network-Based Retrieval of Breast Histopathology Images
Artificial Intelligence Department, Faculty of Computer Engineering, University of Isfahan, Isfahan, Iran；Department of Computer Engineering Techniques, Mazaya University College, Nasiriyah, Iraq；Department of Medicine, Danube Private University, Krems an der Donau, Austria；Austrian Center for Medical Innovation；Research Center for Medical Image Analysis；Artificial Intelligence, Department of Medicine, Danube Private University, Krems an der Donau, Austria
Techno India University, West Bengal（西孟加拉邦Techno印度大学）
In this paper we present a novel method for an automated diagnosis of breast carcinoma through multilevel iterative variational mode decomposition (VMD) and textural features encompassing Zernaike moments, fractal dimension and entropy features namely, Kapoor entropy, Renyi entropy, Yager entropy features are extracted from VMD components. The proposed method considers the histopathological image as a set of multidimensional spatially-evolving signals. ReliefF algorithm is used to select the discriminatory features and statistically most significant features are fed to squares support vector m…
Institute of Cancer Research（癌症研究所）；Royal Marsden Hospital（皇家马斯登医院）
Ki67 is an important biomarker for breast cancer. Classification of positive and negative Ki67 cells in histology slides is a common approach to determine cancer proliferation status. However, there is a lack of generalizable and accurate methods to automate Ki67 scoring in large-scale patient cohorts. In this work, we have employed a novel deep learning technique based on hypercolumn descriptors for cell classification in Ki67 images. Specifically, we developed the Simultaneous Detection and Cell Segmentation (DeepSDCS) network to perform cell segmentation and detection. VGG16 network was use…
IBM Research（IBM研究院）；ETH Zurich（苏黎世联邦理工学院）；EPFL（洛桑联邦理工学院）；IRCCS-Fondazione Pascale（帕斯卡尔IRCCS基金会）；ICAR-CNR（意大利国家研究委员会ICAR研究所）；Aurigen（Aurigen公司）；CHUV（洛桑大学医院）
Cancer diagnosis, prognosis, and therapy response predictions from tissue specimens highly depend on the phenotype and topological distribution of constituting histological entities. Thus, adequate tissue representations for encoding histological entities is imperative for computer aided cancer patient care. To this end, several approaches have leveraged cell-graphs that encode cell morphology and organization to denote the tissue information. These allow for utilizing machine learning to map tissue representations to tissue functionality to help quantify their relationship. Though cellular in…
FEMTO-ST institute（弗朗什-孔泰电子、机械、热学与光学科学与技术研究所）；Univ. Bourgogne Franche-Comte（勃艮第-弗朗什-孔泰大学）；CNRS（法国国家科学研究中心）；ENSMM（国立高等机械与微技术学院）；Beihang University（北京航空航天大学）；Hôpital Nord Franche-Comté（北弗朗什-孔泰医院）；Batna2 University（巴特纳第二大学）
The Ki-67 proliferation index is an essential biomarker that helps pathologists to diagnose and select appropriate treatments. However, automatic evaluation of Ki-67 is difficult due to nuclei overlapping and complex variations in their properties. This paper proposes an integrated pipeline for accurate automatic counting of Ki-67, where the impact of nuclei separation techniques is highlighted. First, semantic segmentation is performed by combining the Squeez and Excitation Resnet and Unet algorithms to extract nuclei from the background. The extracted nuclei are then divided into overlapped …
Automatization of the diagnosis of any kind of disease is of great importance and it's gaining speed as more and more deep learning solutions are applied to different problems. One of such computer aided systems could be a decision support too able to accurately differentiate between different types of breast cancer histological images - normal tissue or carcinoma. In this paper authors present a deep learning solution, based on convolutional capsule network for classification of four types of images of breast tissue biopsy when hematoxylin and eusin staining is applied. The cross-validation a…
University of Saskatchewan（萨斯喀彻温大学）；University of Tulane（杜兰大学）
Breast cancer is one of the most common causes of cancer-related death in women worldwide. Early and accurate diagnosis of breast cancer may significantly increase the survival rate of patients. In this study, we aim to develop a fully automatic, deep learning-based, method using descriptor features extracted by Deep Convolutional Neural Network (DCNN) models and pooling operation for the classification of hematoxylin and eosin stain (H&E) histological breast cancer images provided as a part of the International Conference on Image Analysis and Recognition (ICIAR) 2018 Grand Challenge on BreAs…
Mansoura University（曼苏拉大学）；Higher Institute of Engineering and Technology（高等工程技术学院）
Breast cancer is one of the most serious types of cancer that can occur in women. The automatic diagnosis of breast cancer by analyzing histological images (HIs) is important for patients and their prognosis. The classification of HIs provides clinicians with an accurate understanding of diseases and allows them to treat patients more efficiently. Deep learning (DL) approaches have been successfully employed in a variety of fields, particularly medical imaging, due to their capacity to extract features automatically. This study aims to classify different types of breast cancer using HIs. In th…
A Preliminary Investigation into Search and Matching for Tumour Discrimination in WHO Breast Taxonomy Using Deep Networks
Mayo Clinic（梅奥诊所）；University of Waterloo（滑铁卢大学）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）；Luma Medical Centre（卢玛医疗中心）；International Agency for Research on Cancer（国际癌症研究机构）；Rhazes Lab, Department of Artificial Intelligence and Informatics, Mayo Clinic, Rochester, MN, USA（梅奥诊所Rhazes实验室（人工智能与信息学系））；Kimia Lab, University of Waterloo, Canada（滑铁卢大学Kimia实验室）
Radboud University Medical Center（拉德堡德大学医学中心）；Canisius-Wilhelmina Hospital（卡尼修斯-威廉明娜医院）；Jeroen Bosch Hospital（耶伦博斯医院）
Manual counting of mitotic tumor cells in tissue sections constitutes one of the strongest prognostic markers for breast cancer. This procedure, however, is time-consuming and error-prone. We developed a method to automatically detect mitotic figures in breast cancer tissue sections based on convolutional neural networks (CNNs). Application of CNNs to hematoxylin and eosin (H&E) stained histological tissue sections is hampered by: (1) noisy and expensive reference standards established by pathologists, (2) lack of generalization due to staining variation across laboratories, and (3) high compu…
The size of nuclei in histological preparations from excised breast tumors is predictive of patient outcome (large nuclei indicate poor outcome). Pathologists take into account nuclear size when performing breast cancer grading. In addition, the mean nuclear area (MNA) has been shown to have independent prognostic value. The straightforward approach to measuring nuclear size is by performing nuclei segmentation. We hypothesize that given an image of a tumor region with known nuclei locations, the area of the individual nuclei and region statistics such as the MNA can be reliably computed direc…
Improved Breast Cancer Diagnosis through Transfer Learning on Hematoxylin and Eosin Stained Histology Images
TIB - Leibniz Information Center for Science and Technology（莱布尼茨科学技术信息中心（TIB））；CARESAI；Cairo University（开罗大学）；University of Tasmania（塔斯马尼亚大学）；Obafemi Awolowo University（奥巴费米·阿沃洛沃大学）
National Chiao Tung University（国立交通大学）；Liouying Chi Mei Hospital（柳营奇美医院）；National Pingtung University of Science and Technology（国立屏东科技大学）
Ki-67 is a nuclear protein that can be produced during cell proliferation. The Ki67 index is a valuable prognostic variable in several kinds of cancer. In breast cancer, the index is even routinely checked in many patients. Currently, pathologists use the immunohistochemistry method to calculate the percentage of Ki-67 positive malignant cells as Ki-67 index. The higher score usually means more aggressive tumor behavior. In clinical practice, the measurement of Ki-67 index relies on visual identifying method and manual counting. However, visual and manual assessment method is timeconsuming and…
A Multicentric Dataset for Training and Benchmarking Breast Cancer Segmentation in H&E Slides
Department of Pathology, Radboud University Medical Center（病理学系，拉德堡德大学医学中心）；Fondazione Bruno Kessler（布鲁诺·凯塞勒基金会）；Department of Cellular, Computational and Integrative Biology, University of Trento（细胞、计算与整合生物学系，特伦托大学）；Biopticka Laboratory Ltd.（Biopticka实验室有限公司）；Department of Pathology, Faculty of Medicine in Pilsen, Charles University（病理学系，皮尔森医学院，查尔斯大学）；University of Washington Medical Center（华盛顿大学医学中心）；Department of Surgical Pathology, Santa Chiara Hospital, APSS（外科病理学系，圣查理医院，APSS）；CISMed - Centre for Medical Sciences, University of Trento（CISMed医学研究中心，特伦托大学）；Pathology Unit, University and Hospital Trust of Verona（病理学部，韦罗纳大学与医院信托）；IRCCS Sacro Cuore Don Calabria Hospital（IRCCS圣心多玛利亚医院）
University of Memphis（孟菲斯大学）；Microsoft Research（微软研究院）；AutoML Image Lab（AutoML图像实验室）
Breast Cancer is the most prevalent cancer in the world. The World Health Organization reports that the disease still affects a significant portion of the developing world citing increased mortality rates in the majority of low to middle income countries. The most popular protocol pathologists use for diagnosing breast cancer is the Nottingham grading system which grades the proliferation of tumors based on 3 major criteria, the most important of them being mitotic cell count. The way in which pathologists evaluate mitotic cell count is to subjectively and qualitatively analyze cells present i…
Variational U-Net with Local Alignment for Joint Tumor Extraction and Registration (VALOR-Net) of Breast MRI Data Acquired at Two Different Field Strengths
University of Peshawar（白沙瓦大学）；Danube Private University（多瑙河私立大学）；University of Engineering & Technology, Peshawar（白沙瓦工程与技术大学）；Medical University of Vienna（维也纳医科大学）；Karl Landsteiner Society（卡尔·兰施泰纳学会）；University of Cambridge（剑桥大学）
Predicting Neoadjuvant Chemotherapy Response in Triple-Negative Breast Cancer Using Pre-Treatment Histopathologic Images
The Ohio State University（俄亥俄州立大学）；University of Rochester Medical Center（罗切斯特大学医学中心）；Warren Alpert Medical School of Brown University（布朗大学沃伦·阿尔珀特医学院）；Lifespan Academic Medical Center（Lifespan学术医学中心）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
Stanford School of Medicine（斯坦福医学院）；Center for Artificial Intelligence in Medicine & Imaging（医学与人工智能中心）；Department of Urology（泌尿外科）；Institute for Pathology and Cytology, Schüttorf（许特霍夫病理与细胞研究所）；Department of Radiology（放射科）；Department of Biomedical Data Science（生物医学数据科学系）
Different convolutional neural network (CNN) models have been tested for their application in histological image analyses. However, these models are prone to overfitting due to their large parameter capacity, requiring more data or valuable computational resources for model training. Given these limitations, we introduced a novel architecture (termed PlexusNet). We utilized 310 Hematoxylin and Eosin stained (H&E) annotated histological images of prostate cancer cases from TCGA-PRAD and Stanford University and 398 H&E whole slides images from the Camelyon 2016 challenge. PlexusNet-architecture …
UWarp: A Whole Slide Image Registration Pipeline to Characterize Scanner-Induced Local Domain Shift
Ummon HealthTech SAS；Centre Georges-François Leclerc（乔治-弗朗索瓦·勒克莱尔中心）；ICMUB laboratory, Université de Bourgogne Europe（勃艮第欧洲大学ICMUB实验室）；University Hospital of Dijon（第戎大学医院）；Hospital de Tortosa Verge de la Cinta (HTVC), Catalan Health Institute (ICS)（托尔托萨维尔赫-德拉辛塔医院 (HTVC)，加泰罗尼亚卫生研究所 (ICS)）；Pere Virgili Institute for Health Research (IISPV)（佩雷·维尔吉利健康研究所 (IISPV)）；Universitat Rovira i Virgili (URV)（罗维拉-威尔吉利大学 (URV)）；Nałęcz Institute of Biocybernetics and Biomedical Engineering Polish Academy of Sciences（波兰科学院纳文奇生物控制论与生物医学工程研究所）；Cairo University（开罗大学）；University of Udine（乌迪内大学）
