We evaluate the performance of federated learning (FL) in developing deep learning models for analysis of digitized tissue sections. A classification application was considered as the example use case, on quantifiying the distribution of tumor infiltrating lymphocytes within whole slide images (WSIs). A deep learning classification model was trained using 50*50 square micron patches extracted from the WSIs. We simulated a FL environment in which a dataset, generated from WSIs of cancer from numerous anatomical sites available by The Cancer Genome Atlas repository, is partitioned in 8 different nodes. Our results show that the model trained with the federated training approach achieves similar performance, both quantitatively and qualitatively, to that of a model trained with all the training data pooled at a centralized location. Our study shows that FL has tremendous potential for enabling development of more robust and accurate models for histopathology image analysis without having to collect large and diverse training data at a single location.
Deep learning classifiers for characterization of whole slide tissue morphology require large volumes of annotated data to learn variations across different tissue and cancer types. As is well known, manual generation of digital pathology training data is time consuming and expensive. In this paper, we propose a semi-automated method for annotating a group of similar instances at once, instead of collecting only per-instance manual annotations. This allows for a much larger training set, that reflects visual variability across multiple cancer types and thus training of a single network which c…
Netherlands Cancer Institute（荷兰癌症研究所）；University of Amsterdam（阿姆斯特丹大学）；VU University Amsterdam（阿姆斯特丹自由大学）；Ellogon AI B.V.（Ellogon AI 公司）；Radboud University Medical Center（拉德堡德大学医学中心）
We present WeakSTIL, an interpretable two-stage weak label deep learning pipeline for scoring the percentage of stromal tumor infiltrating lymphocytes (sTIL%) in H&E-stained whole-slide images (WSIs) of breast cancer tissue. The sTIL% score is a prognostic and predictive biomarker for many solid tumor types. However, due to the high labeling efforts and high intra- and interobserver variability within and between expert annotators, this biomarker is currently not used in routine clinical decision making. WeakSTIL compresses tiles of a WSI using a feature extractor pre-trained with self-supervi…
UiT The Arctic University of Norway（挪威北极大学）；University Hospital of North Norway（北挪威大学医院）
Increased levels of tumor infiltrating lymphocytes (TILs) in cancer tissue indicate favourable outcomes in many types of cancer. Manual quantification of immune cells is inaccurate and time consuming for pathologists. Our aim is to leverage a computational solution to automatically quantify TILs in whole slide images (WSIs) of standard diagnostic haematoxylin and eosin stained sections (H&E slides) from lung cancer patients. Our approach is to transfer an open source machine learning method for segmentation and classification of nuclei in H&E slides trained on public data to TIL quantification…
Automating tumor-infiltrating lymphocyte assessment in breast cancer histopathology images using QuPath: a transparent and accessible machine learning pipeline
UiT The Arctic University of Norway（挪威北极圈大学）；University Hospital of North Norway（挪威北部大学医院）
University of Warwick（华威大学）；Queen Mary University of London（伦敦玛丽女王大学）；University of Sheffield（谢菲尔德大学）
The quantification of tumor-infiltrating lymphocytes (TILs) has been shown to be an independent predictor for prognosis of breast cancer patients. Typically, pathologists give an estimate of the proportion of the stromal region that contains TILs to obtain a TILs score. The Tumor InfiltratinG lymphocytes in breast cancER (TiGER) challenge, aims to assess the prognostic significance of computer-generated TILs scores for predicting survival as part of a Cox proportional hazards model. For this challenge, as the TIAger team, we have developed an algorithm to first segment tumor vs. stroma, before…
Shiraz University（设拉子大学）；Koc University（科奇大学）；Shiraz University of Medical Sciences（设拉子医科大学）
The nuclear protein Ki-67 and Tumor infiltrating lymphocytes (TILs) have been introduced as prognostic factors in predicting tumor progression and its treatment response. The value of the Ki-67 index and TILs in approach to heterogeneous tumors such as Breast cancer (BC), known as the most common cancer in women worldwide, has been highlighted in the literature. Due to the indeterminable and subjective nature of Ki-67 as well as TILs scoring, automated methods using machine learning, specifically approaches based on deep learning, have attracted attention. Yet, deep learning methods need consi…
University of Warwick（华威大学）；Shaukat Khanum Memorial Cancer Hospital Research Centre（肖卡特·汗姆纪念癌症医院研究中心）；University of Birmingham（伯明翰大学）；Newcastle University（纽卡斯尔大学）；University of Sheffield（谢菲尔德大学）；The Alan Turing Institute（阿兰·图灵研究所）；University Hospitals Coventry & Warwickshire NHS Trust（考文垂与华威郡大学医院NHS信托）
The infiltration of T-lymphocytes in the stroma and tumour is an indication of an effective immune response against the tumour, resulting in better survival. In this study, our aim is to explore the prognostic significance of tumour-associated stroma infiltrating lymphocytes (TASILs) in head and neck squamous cell carcinoma (HNSCC) through an AI based automated method. A deep learning based automated method was employed to segment tumour, stroma and lymphocytes in digitally scanned whole slide images of HNSCC tissue slides. The spatial patterns of lymphocytes and tumour-associated stroma were …
A Weakly Supervised Segmentation Network Embedding Cross-scale Attention Guidance and Noise-sensitive Constraint for Detecting Tertiary Lymphoid Structures of Pancreatic Tumors
Nanjing University of Science and Technology（南京理工大学）；Nanjing University（南京大学）；Nanjing Drum Tower Hospital（南京鼓楼医院）；Affiliated Hospital of Nanjing University of Chinese Medicine（南京中医药大学附属医院）
Stony Brook University（石溪大学）；Stony Brook Medicine（石溪医学院）；Stony Brook University Hospital（石溪大学医院）；University of Michigan Medical School（密歇根大学医学院）；National Cancer Institute（美国国家癌症研究所）；National Institutes of Health（美国国立卫生研究院）；Emory University（埃默里大学）
Quantitative assessment of Tumor-TIL spatial relationships is increasingly important in both basic science and clinical aspects of breast cancer research. We have developed and evaluated convolutional neural network (CNN) analysis pipelines to generate combined maps of cancer regions and tumor infiltrating lymphocytes (TILs) in routine diagnostic breast cancer whole slide tissue images (WSIs). We produce interactive whole slide maps that provide 1) insight about the structural patterns and spatial distribution of lymphocytic infiltrates and 2) facilitate improved quantification of TILs. We eva…
Indian Institute of Technology Bombay（印度理工学院孟买分校）；University of Illinois（伊利诺伊大学）
Spatial arrangement of cells of various types, such as tumor infiltrating lymphocytes and the advancing edge of a tumor, are important features for detecting and characterizing cancers. However, convolutional neural networks (CNNs) do not explicitly extract intricate features of the spatial arrangements of the cells from histopathology images. In this work, we propose to classify cancers using graph convolutional networks (GCNs) by modeling a tissue section as a multi-attributed spatial graph of its constituent cells. Cells are detected using their nuclei in H&E stained tissue image, and each …
Central South University（中南大学）；Nanjing Tech University（南京工业大学）；The third affiliated hospital of Soochow University（苏州大学第三附属医院）
Digital pathological analysis is run as the main examination used for cancer diagnosis. Recently, deep learning-driven feature extraction from pathology images is able to detect genetic variations and tumor environment, but few studies focus on differential gene expression in tumor cells. In this paper, we propose a self-supervised contrastive learning framework, HistCode, to infer differential gene expressions from whole slide images (WSIs). We leveraged contrastive learning on large-scale unannotated WSIs to derive slide-level histopathological feature in latent space, and then transfer it t…
ECTIL: Label-efficient Computational Tumour Infiltrating Lymphocyte (TIL) assessment in breast cancer: Multicentre validation in 2,340 patients with breast cancer
University of Amsterdam（阿姆斯特丹大学）；Netherlands Cancer Institute（荷兰癌症研究所）；Erasmus University Medical Center（伊拉斯姆斯大学医学中心）；University Medical Center Groningen（格罗宁根大学医学中心）；The Amsterdam University Medical Centre, AUMC, The University of Amsterdam（阿姆斯特丹大学医学中心（阿姆斯特丹大学））；ZAS Hospitals（ZAS医院）；Peter Mac Callum Centre（彼得·麦卡勒姆中心）；University Medical Center Utrecht（乌得勒支大学医学中心）
Histo-Miner: Deep learning based tissue features extraction pipeline from H&E whole slide images of cutaneous squamous cell carcinoma
Faculty of Mathematics and Natural Sciences, University of Cologne（数学与自然科学学院，科隆大学）；Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne（细胞应激反应与年龄相关疾病卓越集群（CECAD），科隆大学）；Department for Dermatology, University Hospital Cologne（皮肤科部门，科隆大学医院）；Department of Dermatology and Allergy, School of Medicine, Technical University of Munich（皮肤科与过敏科部门，医学院，慕尼黑技术大学）；Department of Dermatology, Dortmund Hospital gGmbH, University Witten/Herdecke（皮肤科部门，多特蒙德医院gGmbH，乌尔姆/赫尔德克大学）；Department of Dermatology, Venereology and Allergology, Helios St. Elisabeth Hospital Oberhausen, University Witten/Herdecke（皮肤科、性病科与过敏科部门，Helios圣埃利莎医院奥伯豪森，乌尔姆/赫尔德克大学）；Department of Dermatology and Allergology, University Hospital of the Paracelsus Medical University Salzburg（皮肤科与过敏科部门，帕拉塞尔医学大学萨尔茨堡大学医院）；Department of Dermatology and Allergology, University Hospital Bonn（皮肤科与过敏科部门，波恩大学医院）；Medical Clinic III for Oncology, Hematology, Immune-Oncology and Rheumatology, University Hospital Bonn (UKB)（肿瘤科、血液科、免疫肿瘤科和风湿科第三医疗部门，波恩大学医院（UKB））
In digital pathology, both detection and classification of cells are important for automatic diagnostic and prognostic tasks. Classifying cells into subtypes, such as tumor cells, lymphocytes or stromal cells is particularly challenging. Existing methods focus on morphological appearance of individual cells, whereas in practice pathologists often infer cell classes through their spatial context. In this paper, we propose a novel method for both detection and classification that explicitly incorporates spatial contextual information. We use the spatial statistical function to describe local den…
Osteosarcoma is the most common primary malignant bone tumor, which has high mortality due to easy lung metastasis. Osteosarcoma is a highly anaplastic, pleomorphic tumor with a variety of tumor cell morphology, including fusiform, oval, epithelial, lymphocyte like, small round, transparent cells, etc. Due to the multiple patterns of osteosarcoma cell morphology, pathologists have differences in the classification (viable tumor, necrotic tumor, non-tumor) of osteosarcoma. Therefore, automatic and accurate recognition algorithms can help pathologists greatly reduce time and improve diagnostic a…
Resource-Limited Automated Ki67 Index Estimation in Breast Cancer
Università degli Studi di Milano（米兰大学）；European Commission, Joint Research Centre (JRC)（欧盟委员会联合研究中心）；Humanitas University（人道大学）；IRCCS Humanitas Clinical and Research Hospital（IRCCS人道临床与研究医院）；CINI National Laboratory in Artificial Intelligence and Intelligent Systems（CINI人工智能与智能系统国家实验室）
Histopathological images of tumors contain abundant information about how tumors grow and how they interact with their micro-environment. Better understanding of tissue phenotypes in these images could reveal novel determinants of pathological processes underlying cancer, and in turn improve diagnosis and treatment options. Advances of Deep learning makes it ideal to achieve those goals, however, its application is limited by the cost of high quality labels from patients data. Unsupervised learning, in particular, deep generative models with representation learning properties provides an alter…
