The analysis of the tumor environment on digital histopathology slides is becoming key for the understanding of the immune response against cancer, supporting the development of novel immuno-therapies. We introduce here a novel deep learning solution to the related problem of tumor epithelium segmentation. While most existing deep learning segmentation approaches are trained on time-consuming and costly manual annotation on single stain domain (PD-L1), we leverage here semi-automatically labeled images from a second stain domain (Cytokeratin-CK). We introduce an end-to-end trainable network that jointly segment tumor epithelium on PD-L1 while leveraging unpaired image-to-image translation between CK and PD-L1, therefore completely bypassing the need for serial sections or re-staining of slides. Extending the method to differentiate between PD-L1 positive and negative tumor epithelium regions enables the automated estimation of the PD-L1 Tumor Cell (TC) score. Quantitative experimental results demonstrate the accuracy of our approach against state-of-the-art segmentation methods.
Rutgers University（罗格斯大学）；Rutgers Cancer Institute of New Jersey（新泽西罗格斯癌症研究所）
Automatic and accurate Gleason grading of histopathology tissue slides is crucial for prostate cancer diagnosis, treatment, and prognosis. Usually, histopathology tissue slides from different institutions show heterogeneous appearances because of different tissue preparation and staining procedures, thus the predictable model learned from one domain may not be applicable to a new domain directly. Here we propose to adopt unsupervised domain adaptation to transfer the discriminative knowledge obtained from the source domain to the target domain without requiring labeling of images at the target…
Breast cancer is the most commonly diagnosed cancer worldwide, with over two million new cases each year. During diagnostic tumour grading, pathologists manually count the number of dividing cells (mitotic figures) in biopsy or tumour resection specimens. Since the process is subjective and time-consuming, data-driven artificial intelligence (AI) methods have been developed to automatically detect mitotic figures. However, these methods often generalise poorly, with performance reduced by variations in tissue types, staining protocols, or the scanners used to digitise whole-slide images. Domai…
Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校贝克曼高级科学技术研究所）；Microsoft Research（微软研究院）
Histopathology is critical for the diagnosis of many diseases, including cancer. These protocols typically require pathologists to manually evaluate slides under a microscope, which is time-consuming and subjective, leading to interest in machine learning to automate analysis. However, computational techniques are limited by batch effects, where technical factors like differences in preparation protocol or scanners can alter the appearance of slides, causing models trained on one institution to fail when generalizing to others. Here, we propose a domain adaptation method that improves the gene…
While federated learning is a promising approach for training deep learning models over distributed sensitive datasets, it presents new challenges for machine learning, especially when applied in the medical domain where multi-centric data heterogeneity is common. Building on previous domain adaptation works, this paper proposes a novel federated learning approach for deep learning architectures via the introduction of local-statistic batch normalization (BN) layers, resulting in collaboratively-trained, yet center-specific models. This strategy improves robustness to data heterogeneity while …
The success of deep learning has set new benchmarks for many medical image analysis tasks. However, deep models often fail to generalize in the presence of distribution shifts between training (source) data and test (target) data. One method commonly employed to counter distribution shifts is domain adaptation: using samples from the target domain to learn to account for shifted distributions. In this work we propose an unsupervised domain adaptation approach that uses graph neural networks and, disentangled semantic and domain invariant structural features, allowing for better performance acr…
Shiraz University of Technology（设拉子理工大学）；National Brain Mapping Laboratory（国家脑图谱实验室）
We propose a two-step domain shift-invariant mitosis cell detection method based on Faster RCNN and a convolutional neural network (CNN). We generate various domain-shifted versions of existing histopathology images using a stain augmentation technique, enabling our method to effectively learn various stain domains and achieve better generalization. The performance of our method is evaluated on the preliminary test data set of the MIDOG-2021 challenge. The experimental results demonstrate that the proposed mitosis detection method can achieve promising performance for domain-shifted histopatho…
University of Stavanger（斯塔万格大学）；Erasmus MC Cancer Institute, University Medical Center（伊拉斯姆斯医学中心癌症研究所，大学医学中心）
Accurate segmentation of tissue in histopathological images can be very beneficial for defining regions of interest (ROI) for streamline of diagnostic and prognostic tasks. Still, adapting to different domains is essential for histopathology image analysis, as the visual characteristics of tissues can vary significantly across datasets. Yet, acquiring sufficient annotated data in the medical domain is cumbersome and time-consuming. The labeling effort can be significantly reduced by leveraging active learning, which enables the selective annotation of the most informative samples. Our proposed…
Despite their success in many computer vision tasks, convolutional networks tend to require large amounts of labeled data to achieve generalization. Furthermore, the performance is not guaranteed on a sample from an unseen domain at test time, if the network was not exposed to similar samples from that domain at training time. This hinders the adoption of these techniques in clinical setting where the imaging data is scarce, and where the intra- and inter-domain variance of the data can be substantial. We propose a domain adaptation technique that is especially suitable for deep networks to al…
University of Leeds（利兹大学）；Leeds Teaching Hospitals NHS Trust（利兹教学医院NHS信托基金会）
Weakly-supervised classification of histopathology slides is a computationally intensive task, with a typical whole slide image (WSI) containing billions of pixels to process. We propose Discriminative Region Active Sampling for Multiple Instance Learning (DRAS-MIL), a computationally efficient slide classification method using attention scores to focus sampling on highly discriminative regions. We apply this to the diagnosis of ovarian cancer histological subtypes, which is an essential part of the patient care pathway as different subtypes have different genetic and molecular profiles, treat…
Histopathology image embedding is an active research area in computer vision. Most of the embedding models exclusively concentrate on a specific magnification level. However, a useful task in histopathology embedding is to train an embedding space regardless of the magnification level. Two main approaches for tackling this goal are domain adaptation and domain generalization, where the target magnification levels may or may not be introduced to the model in training, respectively. Although magnification adaptation is a well-studied topic in the literature, this paper, to the best of our knowle…
Mitigating 3D Prostate Biparametric MRI Data Scarcity through Domain Adaptation using Locally-Trained Latent Diffusion Models for Prostate Cancer Detection
Institute of Biomedical Engineering, University of Toronto（多伦多大学生物医学工程研究所）；Lunenfeld-Tanenbaum Research Institute, Mount Sinai Hospital（圣心医院卢内尔-塔内本研究所）；KITE Research Institute, Toronto Rehabilitation Institute, University Health Network（多伦多康复研究所、KITE研究所在大学健康网络）；Joint Department of Medical Imaging, University of Toronto, Princess Margaret Hospital, and Sinai Health systems（多伦多大学联合医学影像部门、玛格丽特医院及辛纳医疗系统）；Department of Computer Science, University of Toronto（多伦多大学计算机科学系）；Faculty Affiliate of the Vector Institute, Toronto（向量研究所教职员工）
IBM Research Europe（IBM欧洲研究院）；ETH Zurich（苏黎世联邦理工学院）；Uppsala University（乌普萨拉大学）
Methods for unsupervised domain adaptation (UDA) help to improve the performance of deep neural networks on unseen domains without any labeled data. Especially in medical disciplines such as histopathology, this is crucial since large datasets with detailed annotations are scarce. While the majority of existing UDA methods focus on the adaptation from a labeled source to a single unlabeled target domain, many real-world applications with a long life cycle involve more than one target domain. Thus, the ability to sequentially adapt to multiple target domains becomes essential. In settings where…
In this work, we propose a mutual information (MI) based unsupervised domain adaptation (UDA) method for the cross-domain nuclei segmentation. Nuclei vary substantially in structure and appearances across different cancer types, leading to a drop in performance of deep learning models when trained on one cancer type and tested on another. This domain shift becomes even more critical as accurate segmentation and quantification of nuclei is an essential histopathology task for the diagnosis/ prognosis of patients and annotating nuclei at the pixel level for new cancer types demands extensive eff…
University of Houston（休斯顿大学）；University Hospital of Cologne（科隆大学医院）；Weill Cornell Medicine（威尔康奈尔医学院）；Cornell University（康奈尔大学）；University of Turin（都灵大学）
Self-supervised learning (SSL) methods are enabling an increasing number of deep learning models to be trained on image datasets in domains where labels are difficult to obtain. These methods, however, struggle to scale to the high resolution of medical imaging datasets, where they are critical for achieving good generalization on label-scarce medical image datasets. In this work, we propose the Histopathology DatasetGAN (HDGAN) framework, an extension of the DatasetGAN semi-supervised framework for image generation and segmentation that scales well to large-resolution histopathology images. W…
Mind the Gap: Continuous Magnification Sampling for Pathology Foundation Models
Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所）；Machine Learning Group, Technische Universität Berlin（柏林技术大学机器学习小组）；Aignostics（Aignostics公司）；Charité, Universitätsmedizin Berlin（柏林夏里特大学医学院）
Domain Adaptation Without the Compute Burden for Efficient Whole Slide Image Analysis
Institute for People-Centered AI, University of Surrey（以人为本的人工智能研究所，萨里大学）；Centre of Vision, Speech and Signal Processing, University of Surrey（视觉、语音和信号处理中心，萨里大学）
Korea University（高丽大学）；University of Warwick（华威大学）
The appearance of histopathology images depends on tissue type, staining and digitization procedure. These vary from source to source and are the potential causes for domain-shift problems. Owing to this problem, despite the great success of deep learning models in computational pathology, a model trained on a specific domain may still perform sub-optimally when we apply them to another domain. To overcome this, we propose a new augmentation called PatchShuffling and a novel self-supervised contrastive learning framework named IMPaSh for pre-training deep learning models. Using these, we obtai…
Efficient Fine-Tuning of DINOv3 Pretrained on Natural Images for Atypical Mitotic Figure Classification
Mines Paris, PSL University, Center for Computational Biology（巴黎 Mines 大学计算生物学中心）；Mines Paris, PSL University, Center for Mathematical Morphology（巴黎 Mines 大学数学形态学中心）；Sanofi（桑福有限公司）；Institut Curie, PSL University（Curie 机构，巴黎 Mines 大学）；INSERM, U1331 Computational Oncology（INSERM，U1331 计算肿瘤学）；Department of Pathology, University Hospital of Nantes（南特大学医院病理学系）
Guangdong Provincial People’s Hospital（广东省人民医院）；Guangdong Academy of Medical Sciences（广东省医学科学院）；Guilin University of Electronic Technology（桂林电子科技大学）；Guangdong Provicial People’s Hospital Ganzhou Hospital（广东省人民医院赣州医院）
Lung cancer is the leading cause of cancer death worldwide, and adenocarcinoma (LUAD) is the most common subtype. Exploiting the potential value of the histopathology images can promote precision medicine in oncology. Tissue segmentation is the basic upstream task of histopathology image analysis. Existing deep learning models have achieved superior segmentation performance but require sufficient pixel-level annotations, which is time-consuming and expensive. To enrich the label resources of LUAD and to alleviate the annotation efforts, we organize this challenge WSSS4LUAD to call for the outs…
ICube, University of Strasbourg, CNRS (UMR 7357)（ICube研究所，斯特拉斯堡大学，CNRS（UMR 7357））；Institute of Pathology, Hannover Medical School（病理研究所，汉诺威医学院）
Semantic segmentation under domain shift remains a fundamental challenge in computer vision, particularly when labelled training data is scarce. This challenge is particularly exemplified in histopathology image analysis, where the same tissue structures must be segmented across images captured under different imaging conditions (stains), each representing a distinct visual domain. Traditional deep learning methods like UNet require extensive labels, which is both costly and time-consuming, particularly when dealing with multiple domains (or stains). To mitigate this, various unsupervised doma…
Histopathology relies on the analysis of microscopic tissue images to diagnose disease. A crucial part of tissue preparation is staining whereby a dye is used to make the salient tissue components more distinguishable. However, differences in laboratory protocols and scanning devices result in significant confounding appearance variation in the corresponding images. This variation increases both human error and the inter-rater variability, as well as hinders the performance of automatic or semi-automatic methods. In the present paper we introduce an unsupervised adversarial network to translat…
Suboptimal generalization of machine learning models on unseen data is a key challenge which hampers the clinical applicability of such models to medical imaging. Although various methods such as domain adaptation and domain generalization have evolved to combat this challenge, learning robust and generalizable representations is core to medical image understanding, and continues to be a problem. Here, we propose STRAP (Style TRansfer Augmentation for histoPathology), a form of data augmentation based on random style transfer from non-medical style source such as artistic paintings, for learni…
RWTH Aachen University（亚琛工业大学）；RWTH Aachen University Hospital（亚琛工业大学医院）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）
In digital pathology, many image analysis tasks are challenged by the need for large and time-consuming manual data annotations to cope with various sources of variability in the image domain. Unsupervised domain adaptation based on image-to-image translation is gaining importance in this field by addressing variabilities without the manual overhead. Here, we tackle the variation of different histological stains by unsupervised stain-to-stain translation to enable a stain-independent applicability of a deep learning segmentation model. We use CycleGANs for stain-to-stain translation in kidney …
UWarp: A Whole Slide Image Registration Pipeline to Characterize Scanner-Induced Local Domain Shift
Ummon HealthTech SAS；Centre Georges-François Leclerc（乔治-弗朗索瓦·勒克莱尔中心）；ICMUB laboratory, Université de Bourgogne Europe（勃艮第欧洲大学ICMUB实验室）；University Hospital of Dijon（第戎大学医院）；Hospital de Tortosa Verge de la Cinta (HTVC), Catalan Health Institute (ICS)（托尔托萨维尔赫-德拉辛塔医院 (HTVC)，加泰罗尼亚卫生研究所 (ICS)）；Pere Virgili Institute for Health Research (IISPV)（佩雷·维尔吉利健康研究所 (IISPV)）；Universitat Rovira i Virgili (URV)（罗维拉-威尔吉利大学 (URV)）；Nałęcz Institute of Biocybernetics and Biomedical Engineering Polish Academy of Sciences（波兰科学院纳文奇生物控制论与生物医学工程研究所）；Cairo University（开罗大学）；University of Udine（乌迪内大学）
