Due to lack of annotated pathological images, transfer learning has been the predominant approach in the field of digital pathology.Pre-trained neural networks based on ImageNet database are often used to extract "off the shelf" features, achieving great success in predicting tissue types, molecular features, and clinical outcomes, etc. We hypothesize that fine-tuning the pre-trained models using histopathological images could further improve feature extraction, and downstream prediction performance.We used 100,000 annotated HE image patches for colorectal cancer (CRC) to finetune a pretrained Xception model via a twostep approach.The features extracted from finetuned Xception (FTX2048) model and Imagepretrained (IMGNET2048) model were compared through: (1) tissue classification for HE images from CRC, same image type that was used for finetuning; (2) prediction of immunerelated gene expression and (3) gene mutations for lung adenocarcinoma (LUAD).Fivefold cross validation was used for model performance evaluation. The extracted features from the finetuned FTX2048 exhibited significantly higher accuracy for predicting tisue types of CRC compared to the off the shelf feature directly from Xception based on ImageNet database. Particularly, FTX2048 markedly improved the accuracy for stroma from 87% to 94%. Similarly, features from FTX2048 boosted the prediction of transcriptomic expression of immunerelated genesin LUAD. For the genes that had signigicant relationships with image fetures, the features fgrom the finetuned model imprroved the prediction for the majority of the genes. Inaddition, fetures from FTX2048 improved prediction of mutation for 5 out of 9 most frequently mutated genes in LUAD.
University of Toronto（多伦多大学）；University of New Brunswick（新不伦瑞克大学）；Sunnybrook Health Sciences Centre（桑尼布鲁克健康科学中心）；St. Michaels Hospital（圣迈克尔医院）；Huron Digital Pathology（休伦数字病理公司）；Kingston Health Sciences Center（金斯顿健康科学中心）
The lack of well-annotated datasets in computational pathology (CPath) obstructs the application of deep learning techniques for classifying medical images. %Since pathologist time is expensive, dataset curation is intrinsically difficult. Many CPath workflows involve transferring learned knowledge between various image domains through transfer learning. Currently, most transfer learning research follows a model-centric approach, tuning network parameters to improve transfer results over few datasets. In this paper, we take a data-centric approach to the transfer learning problem and examine t…
Comparing ImageNet Pre-training with Digital Pathology Foundation Models for Whole Slide Image-Based Survival Analysis
Imperial College London, Department of Electrical and Electronic Engineering, London, UK（帝国理工学院伦敦分校，电子与电气工程系）；Department of Pathology, Institut de Pathologie Multisite, Groupement Hospitalier Sud, Lyon University Hospital, Pierre-Bénite, France（病理学系，多站点病理研究所，南部医院集团，里昂大学医院，皮埃尔-贝内蒂，法国）；University of Lyon, Université Claude Bernard Lyon 1, Lyon, France（里昂大学， Claude Bernard 里昂第一大学，里昂，法国）
Pennsylvania State University（宾夕法尼亚州立大学）；University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）
In histopathological image analysis, feature extraction for classification is a challenging task due to the diversity of histology features suitable for each problem as well as presence of rich geometrical structures. In this paper, we propose an automatic feature discovery framework via learning class-specific dictionaries and present a low-complexity method for classification and disease grading in histopathology. Essentially, our Discriminative Feature-oriented Dictionary Learning (DFDL) method learns class-specific dictionaries such that under a sparsity constraint, the learned dictionarie…
Tianjin University（天津大学）；CSIRO Data61（澳大利亚联邦科学与工业研究组织Data61）；Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局）；Shandong University（山东大学）
Ovarian cancer is one of the most serious cancers that threaten women around the world. Epithelial ovarian cancer (EOC), as the most commonly seen subtype of ovarian cancer, has rather high mortality rate and poor prognosis among various gynecological cancers. Survival analysis outcome is able to provide treatment advices to doctors. In recent years, with the development of medical imaging technology, survival prediction approaches based on pathological images have been proposed. In this study, we designed a deep framework named EOCSA which analyzes the prognosis of EOC patients based on patho…
Stony Brook University（石溪大学）；Brigham and Women's Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Fraunhofer Institute for Medical Image Computing MEVIS（弗劳恩霍夫医学图像计算研究所）；University of Bremen（不来梅大学）
Histopathological assessments, including surgical resection and core needle biopsy, are the standard procedures in the diagnosis of the prostate cancer. Current interpretation of the histopathology images includes the determination of the tumor area, Gleason grading, and identification of certain prognosis-critical features. Such a process is not only tedious, but also prune to intra/inter-observe variabilities. Recently, FDA cleared the marketing of the first whole slide imaging system for digital pathology. This opens a new era for the computer aided prostate image analysis and feature extra…
Transfer learning allows us to exploit knowledge gained from one task to assist in solving another but relevant task. In modern computer vision research, the question is which architecture performs better for a given dataset. In this paper, we compare the performance of 14 pre-trained ImageNet models on the histopathologic cancer detection dataset, where each model has been configured as a naive model, feature extractor model, or fine-tuned model. Densenet161 has been shown to have high precision whilst Resnet101 has a high recall. A high precision model is suitable to be used when follow-up e…
Glioma subtype classification from histopathological images using in-domain and out-of-domain transfer learning: An experimental study
Luxembourg Institute of Health（卢森堡卫生研究院）；Dr. Senckenberg Institute of Neurooncology（森肯伯格神经肿瘤学研究所）；University Hospital Frankfurt（法兰克福大学医院）；Goethe University（歌德大学）；Frankfurt Cancer Institute（法兰克福癌症研究所）；University Cancer Center Frankfurt（法兰克福大学癌症中心）；Laboratoire national de santé（国家卫生实验室）；National Center of Pathology（国家病理中心）；Luxembourg Centre of Neuropathology（卢森堡神经病理学中心）；Saarland University（萨尔大学）
SagivTech Ltd.（SagivTech有限公司）；Roche Innovation Center Munich（慕尼黑罗氏创新中心）
Using features extracted from networks pretrained on ImageNet is a common practice in applications of deep learning for digital pathology. However it presents the downside of missing domain specific image information. In digital pathology, supervised training data is expensive and difficult to collect. We propose a self-supervised method for feature extraction by similarity learning on whole slide images (WSI) that is simple to implement and allows creation of robust and compact image descriptors. We train a siamese network, exploiting image spatial continuity and assuming spatially adjacent t…
In this work, we investigate multi-task learning as a way of pre-training models for classification tasks in digital pathology. It is motivated by the fact that many small and medium-size datasets have been released by the community over the years whereas there is no large scale dataset similar to ImageNet in the domain. We first assemble and transform many digital pathology datasets into a pool of 22 classification tasks and almost 900k images. Then, we propose a simple architecture and training scheme for creating a transferable model and a robust evaluation and selection protocol in order t…
Northeastern University（东北大学）；China Medical University（中国医科大学）；University of Luebeck（吕贝克大学）
The gold standard for gastric cancer detection is gastric histopathological image analysis, but there are certain drawbacks in the existing histopathological detection and diagnosis. In this paper, based on the study of computer aided diagnosis system, graph based features are applied to gastric cancer histopathology microscopic image analysis, and a classifier is used to classify gastric cancer cells from benign cells. Firstly, image segmentation is performed, and after finding the region, cell nuclei are extracted using the k-means method, the minimum spanning tree (MST) is drawn, and graph …
Harnessing Transformers: A Leap Forward in Lung Cancer Image Detection
Laboratoire LITAN École supérieure en Sciences et Technologies de l’Informatique et du Numérique（高等计算机与数字科学技术学院 LITAN 实验室）；College of Engineering and Information Technology, University of Dubai（迪拜大学工程与信息技术学院）；Department of Computer Science, University of Sharjah（沙迦大学计算机科学系）；Institute of Artificial Intelligence, De Montfort University（德蒙福特大学人工智能研究所）
University of Saskatchewan（萨斯喀彻温大学）；University of Tulane（杜兰大学）
Breast cancer is one of the leading causes of death across the world in women. Early diagnosis of this type of cancer is critical for treatment and patient care. Computer-aided detection (CAD) systems using convolutional neural networks (CNN) could assist in the classification of abnormalities. In this study, we proposed an ensemble deep learning-based approach for automatic binary classification of breast histology images. The proposed ensemble model adapts three pre-trained CNNs, namely VGG19, MobileNet, and DenseNet. The ensemble model is used for the feature representation and extraction s…
Northwestern Polytechnical University（西北工业大学）；Tianjin Medical University（天津医科大学）
Goal: Squamous cell carcinoma of cervix is one of the most prevalent cancer worldwide in females. Traditionally, the most indispensable diagnosis of cervix squamous carcinoma is histopathological assessment which is achieved under microscope by pathologist. However, human evaluation of pathology slide is highly depending on the experience of pathologist, thus big inter- and intra-observer variability exists. Digital pathology, in combination with deep learning provides an opportunity to improve the objectivity and efficiency of histopathologic slide analysis. Methods: In this study, we obtaine…
Advancement in digital pathology and artificial intelligence has enabled deep learning-based computer vision techniques for automated disease diagnosis and prognosis. However, WSIs present unique computational and algorithmic challenges. WSIs are gigapixel-sized, making them infeasible to be used directly for training deep neural networks. Hence, for modeling, a two-stage approach is adopted: Patch representations are extracted first, followed by the aggregation for WSI prediction. These approaches require detailed pixel-level annotations for training the patch encoder. However, obtaining thes…
Computer aided diagnosis (CAD) of Breast Cancer (BRCA) images has been an active area of research in recent years. The main goals of this research is to develop reliable automatic methods for detecting and diagnosing different types of BRCA from diagnostic images. In this paper, we present a review of the state of the art CAD methods applied to magnetic resonance (MRI) and mammography images of BRCA patients. The review aims to provide an extensive introduction to different features extracted from BRCA images through texture and statistical analysis and to categorize deep learning frameworks a…
Jagannath University（贾甘纳特大学）；RMIT University（皇家墨尔本理工大学）；The University of Queensland（昆士兰大学）
Cancer is a fatal disease caused by a combination of genetic diseases and a variety of biochemical abnormalities. Lung and colon cancer have emerged as two of the leading causes of death and disability in humans. The histopathological detection of such malignancies is usually the most important component in determining the best course of action. Early detection of the ailment on either front considerably decreases the likelihood of mortality. Machine learning and deep learning techniques can be utilized to speed up such cancer detection, allowing researchers to study a large number of patients…
Cancer is the uncontrollable cell division of abnormal cells inside the human body, which can spread to other body organs. It is one of the non-communicable diseases (NCDs) and NCDs accounts for 71% of total deaths worldwide whereas lung cancer is the second most diagnosed cancer after female breast cancer. Cancer survival rate of lung cancer is only 19%. There are various methods for the diagnosis of lung cancer, such as X-ray, CT scan, PET-CT scan, bronchoscopy and biopsy. However, to know the subtype of lung cancer based on the tissue type H and E staining is widely used, where the staining…
Central South University（中南大学）；Nanjing Tech University（南京工业大学）；The third affiliated hospital of Soochow University（苏州大学第三附属医院）
Digital pathological analysis is run as the main examination used for cancer diagnosis. Recently, deep learning-driven feature extraction from pathology images is able to detect genetic variations and tumor environment, but few studies focus on differential gene expression in tumor cells. In this paper, we propose a self-supervised contrastive learning framework, HistCode, to infer differential gene expressions from whole slide images (WSIs). We leveraged contrastive learning on large-scale unannotated WSIs to derive slide-level histopathological feature in latent space, and then transfer it t…
MB-DSMIL-CL-PL: Scalable Weakly Supervised Ovarian Cancer Subtype Classification and Localisation Using Contrastive and Prototype Learning with Frozen Patch Features
School of Computing Sciences, University of East Anglia（东安格利亚大学计算机科学学院）；Norfolk and Norwich University Hospital（诺福克和诺里奇大学医院）；University of East Anglia（东安格利亚大学）
Department of Applied Tumor Biology, Institute of Pathology, Heidelberg University Hospital（海德堡大学医院病理学研究所应用肿瘤生物学系）；National Center for Hereditary Tumor Syndromes, University Hospital Bonn（波恩大学医院遗传性肿瘤综合征国家中心）；Department of Internal Medicine I, University Hospital Bonn（波恩大学医院第一内科）；Leibniz Institut für Wissensmedien（莱布尼茨知识媒体研究所）
FUJIFILM Corporation（富士胶片公司）；AWS AI Labs（AWS人工智能实验室）；Massachusetts Institute of Technology（麻省理工学院）；Stanford University（斯坦福大学）
Learning visual representations of medical images (e.g., X-rays) is core to medical image understanding but its progress has been held back by the scarcity of human annotations. Existing work commonly relies on fine-tuning weights transferred from ImageNet pretraining, which is suboptimal due to drastically different image characteristics, or rule-based label extraction from the textual report data paired with medical images, which is inaccurate and hard to generalize. Meanwhile, several recent studies show exciting results from unsupervised contrastive learning from natural images, but we fin…
Colorectal liver metastasis is one of most aggressive liver malignancies. While the definition of lesion type based on CT images determines the diagnosis and therapeutic strategy, the discrimination between cancerous and non-cancerous lesions are critical and requires highly skilled expertise, experience and time. In the present work we introduce an end-to-end deep learning approach to assist in the discrimination between liver metastases from colorectal cancer and benign cysts in abdominal CT images of the liver. Our approach incorporates the efficient feature extraction of InceptionV3 combin…
University of Pennsylvania（宾夕法尼亚大学）；Yantai University（烟台大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The Children's Hospital of Philadelphia（费城儿童医院）
Classification of ultrasound (US) kidney images for diagnosis of congenital abnormalities of the kidney and urinary tract (CAKUT) in children is a challenging task. It is desirable to improve existing pattern classification models that are built upon conventional image features. In this study, we propose a transfer learning-based method to extract imaging features from US kidney images in order to improve the CAKUT diagnosis in children. Particularly, a pre-trained deep learning model (imagenet-caffe-alex) is adopted for transfer learning-based feature extraction from 3-channel feature maps co…
Mapping the landscape of histomorphological cancer phenotypes using self-supervised learning on unlabeled, unannotated pathology slides
University of Glasgow（格拉斯哥大学）；New York University School of Medicine（纽约大学医学院）；New York University Langone Health（纽约大学朗格尼医学中心）；Karolinska Institutet（卡罗林斯卡学院）；University College London Hospital（伦敦大学学院医院）
CONNECT Centre, Trinity College Dublin（都柏林圣三一大学CONNECT中心）；University of Sindh（信德大学）；Signy Advanced Technologies（西格尼先进技术公司）；Indian Institute of Technology, Dehli（印度理工学院德里分校）；King Faisal University（费萨尔国王大学）
The current COVID-19 pandemic has motivated the researchers to use artificial intelligence techniques for a potential alternative to reverse transcription-polymerase chain reaction (RT-PCR) due to the limited scale of testing. The chest X-ray (CXR) is one of the alternatives to achieve fast diagnosis but the unavailability of large-scale annotated data makes the clinical implementation of machine learning-based COVID detection difficult. Another issue is the usage of ImageNet pre-trained networks which does not extract reliable feature representations from medical images. In this paper, we pro…
Handling Missing Modalities in Multimodal Survival Prediction for Non-Small Cell Lung Cancer
Unit of Artificial Intelligence and Computer Systems, Department of Engineering（人工智能与计算机系统单位，工程系）；Università Campus Bio-Medico di Roma（罗马生物医学大学）；Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering（诊断与介入部门，辐射物理，生物医学工程）；Umeå University（乌梅学院）；Research Unit of Radiation Oncology, Department of Medicine and Surgery（放射肿瘤研究单位，医学与外科部门）；Fondazione Policlinico Universitario Campus Bio-Medico（生物医学大学附属医院基金会）；Anatomical Pathology Operative Research Unit（解剖病理学操作研究单位）；Research Unit of Anatomical Pathology, Department of Medicine and Surgery（解剖病理学研究单位，医学与外科部门）；Department of Medicine and Surgery（医学与外科部门）
Efficient Feature Extraction Using Light-Weight CNN Attention-Based Deep Learning Architectures for Ultrasound Fetal Plane Classification
Amrita School of Artificial Intelligence（阿米特学校人工智能学院）；Amrita Vishwa Vidyapeetham（阿米特世界大学）；Center for Artificial Intelligence（人工智能中心）；Prince Mohammed Bin Fahd University（普里姆·穆罕默德·本·法赫德大学）
