Objective: We develop a computer-aided diagnosis (CAD) system using deep learning approaches for lesion detection and classification on whole-slide images (WSIs) with breast cancer. The deep features being distinguishing in classification from the convolutional neural networks (CNN) are demonstrated in this study to provide comprehensive interpretability for the proposed CAD system using pathological knowledge. Methods: In the experiment, a total of 186 slides of WSIs were collected and classified into three categories: Non-Carcinoma, Ductal Carcinoma in Situ (DCIS), and Invasive Ductal Carcinoma (IDC). Instead of conducting pixel-wise classification into three classes directly, we designed a hierarchical framework with the multi-view scheme that performs lesion detection for region proposal at higher magnification first and then conducts lesion classification at lower magnification for each detected lesion. Results: The slide-level accuracy rate for three-category classification reaches 90.8% (99/109) through 5-fold cross-validation and achieves 94.8% (73/77) on the testing set. The experimental results show that the morphological characteristics and co-occurrence properties learned by the deep learning models for lesion classification are accordant with the clinical rules in diagnosis. Conclusion: The pathological interpretability of the deep features not only enhances the reliability of the proposed CAD system to gain acceptance from medical specialists, but also facilitates the development of deep learning frameworks for various tasks in pathology. Significance: This paper presents a CAD system for pathological image analysis, which fills the clinical requirements and can be accepted by medical specialists with providing its interpretability from the pathological perspective.
FECT: Classification of Breast Cancer Pathological Images Based on Fusion Features
Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School（清华深圳国际研究生院生物制药与健康工程研究所）；School of Medicine, Nankai University（南开大学医学院）
The difficulty of detecting mitosis and its similarity to non-mitosis objects has remained a challenge in computational pathology. The lack of publicly available data has added more complexity. Deep learning algorithms have shown potentials in mitosis detection tasks. However, they face challenges when applied to pathology images with dense medium and diverse dataset. This paper introduces an optimized UV-Net architecture, developed to focus on mitosis details with high-resolution through feature preservation. Stain normalization methods are used to generalize the trained network. An F1 score …
We proposed a novel convolutional restricted Boltzmann machine CRBM-based radiomic method for predicting pathologic complete response (pCR) to neoadjuvant chemotherapy treatment (NACT) in breast cancer. The method consists of extracting semantic features from CRBM network, and pCR prediction. It was evaluated on the dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) data of 57 patients and using the area under the receiver operating characteristic curve (AUC). Traditional radiomics features and the semantic features learned from CRBM network were extracted from the images acquired …
IBM Research Africa（IBM非洲研究院）；National Cancer Registry（国家癌症登记处）
Like most global cancer registries, the National Cancer Registry in South Africa employs expert human coders to label pathology reports using appropriate International Classification of Disease for Oncology (ICD-O) codes spanning 42 different cancer types. The annotation is extensive for the large volume of cancer pathology reports the registry receives annually from public and private sector institutions. This manual process, coupled with other challenges results in a significant 4-year lag in reporting of annual cancer statistics in South Africa. We present a hierarchical deep learning ensem…
Pathological area segmentation in cardiac magnetic resonance (MR) images plays a vital role in the clinical diagnosis of cardiovascular diseases. Because of the irregular shape and small area, pathological segmentation has always been a challenging task. We propose an anatomy prior based framework, which combines the U-net segmentation network with the attention technique. Leveraging the fact that the pathology is inclusive, we propose a neighborhood penalty strategy to gauge the inclusion relationship between the myocardium and the myocardial infarction and no-reflow areas. This neighborhood …
Institute of Pathology, University Hospital Carl Gustav Carus (UKD), TU Dresden（德累斯顿工业大学卡尔·古斯塔夫·卡鲁斯大学医院病理研究所）；ASGEN GmbH & Co. KG（ASGEN两合有限责任公司）；National Center for Tumor Diseases (NCT), Partner Site Dresden（国家肿瘤疾病中心（德累斯顿合作站点））；ContextVision AB（康泰视伦公司）；Institute for Medical Informatics and Biometry (IMB), Carl Gustav Carus Faculty of Medicine, TU Dresden（德累斯顿工业大学卡尔·古斯塔夫·卡鲁斯医学院医学信息学与生物测量学研究所）；Institute of Bioengineering, School of Life Sciences, EPFL（洛桑联邦理工学院生命科学学院生物工程研究所）；Max Planck Institute of Molecular Cell Biology and Genetics (MPI-CBG)（马克斯·普朗克分子细胞生物学与遗传学研究所）；Center for Systems Biology Dresden (CSBD)（德累斯顿系统生物学中心）
Histo-pathological diagnostics are an inherent part of the everyday work but are particularly laborious and associated with time-consuming manual analysis of image data. In order to cope with the increasing diagnostic case numbers due to the current growth and demographic change of the global population and the progress in personalized medicine, pathologists ask for assistance. Profiting from digital pathology and the use of artificial intelligence, individual solutions can be offered (e.g. detect labeled cancer tissue sections). The testing of the human epidermal growth factor receptor 2 (HER…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Xiangya Hospital, Central South University（中南大学湘雅医院）；University of Chinese Academy of Sciences（中国科学院大学）；Central South University（中南大学）
Molecular subtypes of breast cancer are important references to personalized clinical treatment. For cost and labor savings, only one of the patient's paraffin blocks is usually selected for subsequent immunohistochemistry (IHC) to obtain molecular subtypes. Inevitable sampling error is risky due to tumor heterogeneity and could result in a delay in treatment. Molecular subtype prediction from conventional H&E pathological whole slide images (WSI) using AI method is useful and critical to assist pathologists pre-screen proper paraffin block for IHC. It's a challenging task since only WSI level…
The analysis of FFPE tissue sections stained with haematoxylin and eosin (H&E) or immunohistochemistry (IHC) is an essential part of the pathologic assessment of surgically resected breast cancer specimens. IHC staining has been broadly adopted into diagnostic guidelines and routine workflows to manually assess status and scoring of several established biomarkers, including ER, PGR, HER2 and KI67. However, this is a task that can also be facilitated by computational pathology image analysis methods. The research in computational pathology has recently made numerous substantial advances, often …
Towards a Comprehensive Benchmark for Pathological Lymph Node Metastasis in Breast Cancer Sections
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；National Cancer Center/National Clinical Research Center for Cancer/Cancer Hospital & Shenzhen Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院肿瘤医院（国家癌症中心/国家临床肿瘤医学中心）深圳医院）；Research Institute of Tsinghua, Guangzhou（清华大学广州研究院）
Charles Stark Draper Laboratory（查尔斯·斯塔克·德拉普尔实验室）；Stanford Medicine（斯坦福医学）；Bracco Diagnostics Corporation（布拉科诊断公司）
Molecularly targeted contrast enhanced ultrasound (mCEUS) is a clinically promising approach for early cancer detection through targeted imaging of VEGFR2 (KDR) receptors. We have developed computational enhancement techniques for mCEUS tailored to address the unique challenges of imaging contrast accumulation in humans. These techniques utilize dynamic analysis to distinguish molecularly bound contrast agent from other contrast-mode signal sources, enabling analysis of contrast agent accumulation to be performed during contrast bolus arrival when the signal due to molecular binding is stronge…
Advancing H&E-to-IHC Stain Translation in Breast Cancer: A Multi-Magnification and Attention-Based Approach
Fudan University（复旦大学）；College of Electronic Engineering, Ocean University of China（中国海洋大学电子工程学院）；Research Center for Frontier Fundamental Studies, Zhejiang Lab（之江实验室前沿基础研究中心）；Institutes of Biomedical Sciences, Fudan University（复旦大学生物医学研究院）
Beyond H&E: Unlocking Pathological Insights with Polarization Imaging
The Hong Kong University of Science and Technology（香港科技大学）；Beijing Institute of Collaborative Innovation（北京协同创新研究院）；Peking University Health Science Center, Peking University Third Hospital（北京大学人民医院）；University of Oxford（牛津大学）；Peking University（北京大学）
Central South University（中南大学）；Shaanxi Provincial People's Hospital（陕西省人民医院）
Background. Digital pathology has aroused widespread interest in modern pathology. The key of digitalization is to scan the whole slide image (WSI) at high magnification. The lager the magnification is, the richer details WSI will provide, but the scanning time is longer and the file size of obtained is larger. Methods. We design a strategy to scan slides with low resolution (5X) and a super-resolution method is proposed to restore the image details when in diagnosis. The method is based on a multi-scale generative adversarial network, which sequentially generates three high-resolution images …
Experts' cognition-driven ensemble deep learning for external validation of predicting pathological complete response to neoadjuvant chemotherapy from histological images in breast cancer
Sichuan University（四川大学）；West China Hospital（华西医院）；Zhongjiu Flash Medical Technology Co., Ltd.（中久闪医医疗科技有限公司）；Shanxi Provincial Cancer Hospital（山西省肿瘤医院）；Sichuan Provincial People’s Hospital（四川省人民医院）；The Affiliated Hospital of Southwest Medical University（西南医科大学附属医院）
Unlocking adaptive digital pathology through dynamic feature learning
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；University of Oxford（牛津大学）；Fourth Military Medical University（第四军医大学）；Third Military Medical University（第三军医大学）；Chongqing Advanced Pathology Research Institute（重庆先进病理研究院）；Jinfeng Laboratory（金凤实验室）；Soochow University（苏州大学）；The Affiliated Cancer Hospital of Zhengzhou University and Henan Cancer Hospital（郑州大学附属肿瘤医院暨河南省肿瘤医院）；Research Institute of Tsinghua, Pearl River Delta（清华大学珠三角研究院）
Central South University（中南大学）；Shaanxi Provincial People’s Hospital（陕西省人民医院）；Xiangya Hospital, Central South University（中南大学湘雅医院）
[Purpose] The pathology is decisive for disease diagnosis, but relies heavily on the experienced pathologists. Recently, pathological artificial intelligence (PAI) is thought to improve diagnostic accuracy and efficiency. However, the high performance of PAI based on deep learning in the laboratory generally cannot be reproduced in the clinic. [Methods] Because the data preparation is important for PAI, the paper has reviewed PAI-related studies in the PubMed database published from January 2017 to February 2022, and 118 studies were included. The in-depth analysis of methods for preparing dat…
Transfer Learning Strategies for Pathological Foundation Models: A Systematic Evaluation in Brain Tumor Classification
Hokkaido University（北海道大学）；Hokkaido University Hospital（北海道大学医院）；Nakamura Memorial Hospital（中村纪念医院）；Sapporo Azabu Neurosurgical Hospital（札幌麻布神经外科医院）；Sapporo Hakuyokai Hospital（札幌白洋会医院）；Institute for Chemical Reaction Design and Discovery (WPI-ICReDD)（化学反应设计与发现研究所（WPI-ICReDD））
ERCPMP: An Endoscopic Image and Video Dataset for Colorectal Polyps Morphology and Pathology
Gastroenterology and Liver Disease Research Center, Research Institute for Gastroenterology and Liver Diseases, Shahid Beheshti University of Medical Sciences（沙希德贝赫什提医科大学胃肠病学与肝病研究所胃肠病学与肝病研究中心）；Alborz University of Medical Sciences（厄尔布尔士医科大学）；Shahid Beheshti University of Medical Sciences（沙希德贝赫什提医科大学）；Iran University of Medical Sciences（伊朗医科大学）
Transition from conventional to digital pathology requires a new category of biomedical informatic infrastructure which could facilitate delicate pathological routine. Pathological diagnoses are sensitive to many external factors and is known to be subjective. Only systems that can meet strict requirements in pathology would be able to run along pathological routines and eventually digitized the study area, and the developed platform should comply with existing pathological routines and international standards. Currently, there are a number of available software tools which can perform histopa…
Assessing and Enhancing Robustness of Deep Learning Models with Corruption Emulation in Digital Pathology
College of Engineering, Peking University（北京大学工学院）；Third Hospital, Peking University Health Science Center（北京大学医学部第三医院）；Institute of Biomedical Engineering, Beijing Institute of Collaborative Innovation（北京协同创新研究院生物医学工程研究所）
Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directions
The Hong Kong University of Science and Technology（香港科技大学）；Stanford University School of Medicine（斯坦福大学医学院）；The Chinese University of Hong Kong（香港中文大学）；The University of Hong Kong（香港大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深圳-香港协同创新研究院）
Genentech（基因泰克）；University of Central Florida（中佛罗里达大学）
The integration of multi-modal data, such as pathological images and genomic data, is essential for understanding cancer heterogeneity and complexity for personalized treatments, as well as for enhancing survival predictions. Despite the progress made in integrating pathology and genomic data, most existing methods cannot mine the complex inter-modality relations thoroughly. Additionally, identifying explainable features from these models that govern preclinical discovery and clinical prediction is crucial for cancer diagnosis, prognosis, and therapeutic response studies. We propose PONET- a n…
Breast Cancer Image Classification Method Based on Deep Transfer Learning
The Hong Kong University of Science and Technology（香港科技大学）；University of Southampton（南安普顿大学）；Trine University（特莱恩大学）；Southwest Jiaotong University（西南交通大学）
Breast cancer is the most common cancer among women worldwide. The human epidermal growth factor receptor 2 (HER2) with immunohistochemical (IHC) is widely used for pathological evaluation to provide the appropriate therapy for patients with breast cancer. However, the deficiency of pathologists and subjective and susceptible to inter-observer variation of visual diagnosis are the main challenges. Recently, with the rapid development of artificial intelligence (AI) in disease diagnosis, several automated HER2 scoring methods using traditional computer vision or machine learning methods indicat…
Tencent AI Lab（腾讯AI实验室）；University of Connecticut（康涅狄格大学）；The Fourth Hospital of Hebei Medical University（河北医科大学第四医院）；West China Hospital, Sichuan University（四川大学华西医院）
Artificial Intelligence (AI)-powered pathology is a revolutionary step in the world of digital pathology and shows great promise to increase both diagnosis accuracy and efficiency. However, defocus and motion blur can obscure tissue or cell characteristics hence compromising AI algorithms'accuracy and robustness in analyzing the images. In this paper, we demonstrate a deep-learning-based approach that can alleviate the defocus and motion blur of a microscopic image and output a sharper and cleaner image with retrieved fine details without prior knowledge of the blur type, blur extent and patho…
University of Toronto（多伦多大学）；National Cancer Research Centre（国家癌症研究中心）；Institute for Oncology and Radiology（肿瘤与放射学研究所）
Accurate diagnosis of breast cancer in histopathology images is challenging due to the heterogeneity of cancer cell growth as well as of a variety of benign breast tissue proliferative lesions. In this paper, we propose a practical and self-interpretable invasive cancer diagnosis solution. With minimum annotation information, the proposed method mines contrast patterns between normal and malignant images in unsupervised manner and generates a probability map of abnormalities to verify its reasoning. Particularly, a fully convolutional autoencoder is used to learn the dominant structural patter…
We present BrainPainter, a software that automatically generates images of highlighted brain structures given a list of numbers corresponding to the output colours of each region. Compared to existing visualisation software (i.e. Freesurfer, SPM, 3D Slicer), BrainPainter has three key advantages: (1) it does not require the input data to be in a specialised format, allowing BrainPainter to be used in combination with any neuroimaging analysis tools, (2) it can visualise both cortical and subcortical structures and (3) it can be used to generate movies showing dynamic processes, e.g. propagatio…
