Ali Mammadov, Loïc Le Folgoc, Guillaume Hocquet, Pietro Gori
备注：
MICCAI 2025 - This preprint has not undergone peer review or any post-submission improvements or corrections. The Version of Record of this article is published in LNCS, Springer
数字病理学通过将组织样本数字化为全切片图像（WSIs）彻底改变了该领域。然而，WSIs 的高分辨率和巨大尺寸给深度学习模型的应用带来了显著挑战。作为一种解决方案，WSIs 通常被分割成较小的图像块，并为每张切片赋予一个全局标签（即诊断标签），而不是进行（过于）昂贵的逐像素标注。通过将每张切片视为一个图像块包，多实例学习（MIL）方法已成为 WSI 分类的合适解决方案。MIL 方法的一个主要缺点是其在不同运行中的性能变异性很高，在测试集上的差异可达 10-15 个 AUC 点，这使得可靠地比较不同的 MIL 方法变得困难。这种变异性主要来自三个因素：i）权重初始化，ii）批次（打乱）顺序，iii）学习率。为了解决这一问题，我们引入了一种面向 MIL 方法的多保真度模型融合策略。我们首先训练多个模型若干个 epoch，并根据验证分数对最稳定且最有前景的模型进行平均。该方法可应用于任何现有的 MIL 模型以降低性能变异性。它还能简化超参数调优并提高可复现性，同时保持计算效率。我们使用 2 个不同的数据集、3 种初始化策略和 5 种 MIL 方法，在 WSI 分类任务上对我们的方法进行了广泛验证，总计进行了超过 2000 次实验。
英文摘要：
Digital pathology has revolutionized the field by enabling the digitization of tissue samples into whole slide images (WSIs). However, the high resolution and large size of WSIs present significant challenges when it comes to applying Deep Learning models. As a solution, WSIs are often divided into smaller patches with a global label (\textit{i.e., diagnostic}) per slide, instead of a (too) costly pixel-wise annotation. By treating each slide as a bag of patches, Multiple Instance Learning (MIL) methods have emerged as a suitable solution for WSI classification. A major drawback of MIL methods is their high variability in performance across different runs, which can reach up to 10-15 AUC points on the test set, making it difficult to compare different MIL methods reliably. This variability mainly comes from three factors: i) weight initialization, ii) batch (shuffling) ordering, iii) and learning rate. To address that, we introduce a Multi-Fidelity, Model Fusion strategy for MIL methods. We first train multiple models for a few epochs and average the most stable and promising ones based on validation scores. This approach can be applied to any existing MIL model to reduce performance variability. It also simplifies hyperparameter tuning and improves reproducibility while maintaining computational efficiency. We extensively validate our approach on WSI classification tasks using 2 different datasets, 3 initialization strategies and 5 MIL methods, for a total of more than 2000 experiments.
When designing a diagnostic model for a clinical application, it is crucial to guarantee the robustness of the model with respect to a wide range of image corruptions. Herein, an easy-to-use benchmark is established to evaluate how deep neural networks perform on corrupted pathology images. Specifically, corrupted images are generated by injecting nine types of common corruptions into validation images. Besides, two classification and one ranking metrics are designed to evaluate the prediction and confidence performance under corruption. Evaluated on two resulting benchmark datasets, we find t…
SagivTech Ltd.（SagivTech有限公司）；Roche Innovation Center Munich（慕尼黑罗氏创新中心）
Using features extracted from networks pretrained on ImageNet is a common practice in applications of deep learning for digital pathology. However it presents the downside of missing domain specific image information. In digital pathology, supervised training data is expensive and difficult to collect. We propose a self-supervised method for feature extraction by similarity learning on whole slide images (WSI) that is simple to implement and allows creation of robust and compact image descriptors. We train a siamese network, exploiting image spatial continuity and assuming spatially adjacent t…
Kharkevich Institute for Information Transmission Problems（信息传输问题研究所（Kharkevich研究所））；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
In the last years, neural networks have proven to be a powerful framework for various image analysis problems. However, some application domains have specific limitations. Notably, digital pathology is an example of such fields due to tremendous image sizes and quite limited number of training examples available. In this paper, we adopt state-of-the-art convolutional neural networks (CNN) architectures for digital pathology images analysis. We propose to classify image patches to increase effective sample size and then to apply an ensembling technique to build prediction for the original image…
GAS-MIL: Group-Aggregative Selection Multi-Instance Learning for Ensemble of Foundation Models in Digital Pathology Image Analysis
The University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）；UT Southwestern Medical Center（德克萨斯大学西南医学中心）；Simmons Comprehensive Cancer Center（西蒙斯综合癌症中心）
Systematic Evaluation of Preprocessing Techniques for Accurate Image Registration in Digital Pathology
Institute of Physical Chemistry (IPC) and Abbe Center of Photonics (ACP), Friedrich Schiller University Jena（耶拿弗里德里希·席勒大学物理化学研究所与阿贝光子学中心）；Leibniz Institute of Photonic Technology（莱布尼茨光子技术研究所）
University of Guelph（圭尔夫大学）；Vector Institute（向量研究所）
In Digital Pathology (DP), labeled data is generally very scarce due to the requirement that medical experts provide annotations. We address this issue by learning transferable features from weakly labeled data, which are collected from various parts of the body and are organized by non-medical experts. In this paper, we show that features learned from such weakly labeled datasets are indeed transferable and allow us to achieve highly competitive patch classification results on the colorectal cancer (CRC) dataset [1] and the PatchCamelyon (PCam) dataset [2] while using an order of magnitude le…
Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are…
Xi’an Jiaotong University（西安交通大学）；Pennsylvania State University（宾夕法尼亚州立大学）；the First Affiliated Hospital of Xi’an Jiaotong University（西安交通大学第一附属医院）
Digital pathology plays a crucial role in the development of artificial intelligence in the medical field. The digital pathology platform can make the pathological resources digital and networked, and realize the permanent storage of visual data and the synchronous browsing processing without the limitation of time and space. It has been widely used in various fields of pathology. However, there is still a lack of an open and universal digital pathology platform to assist doctors in the management and analysis of digital pathological sections, as well as the management and structured descripti…
Detecting Domain Shift in Multiple Instance Learning for Digital Pathology Using Fréchet Domain Distance
Center for Medical Imaging Science and Visualization, Linköping University, University Hospital（林雪平大学 医学影像科学与可视化中心，大学医院）；Department of Science and Technology, Linköping University（林雪平大学 科学技术系）；Department of Biomedical and Clinical Sciences, Linköping University（林雪平大学 生物医学与临床科学系）；Sectra AB（塞克特拉公司）
EPFL（洛桑联邦理工学院）；IBM Research Zürich（IBM苏黎世研究院）；ETH Zürich（苏黎世联邦理工学院）；National Cancer Institute - IRCCS-Fondazione Pascale（国家癌症研究所 - IRCCS帕斯卡尔基金会）
Explainability of machine learning (ML) techniques in digital pathology (DP) is of great significance to facilitate their wide adoption in clinics. Recently, graph techniques encoding relevant biological entities have been employed to represent and assess DP images. Such paradigm shift from pixel-wise to entity-wise analysis provides more control over concept representation. In this paper, we introduce a post-hoc explainer to derive compact per-instance explanations emphasizing diagnostically important entities in the graph. Although we focus our analyses to cells and cellular interactions in …
Linköping University（林雪平大学）；Center for Medical Image Science and Visualization, Linköping University（林雪平大学医学影像科学与可视化中心）；Sectra AB
Artificial intelligence (AI) has shown great promise for diagnostic imaging assessments. However, the application of AI to support medical diagnostics in clinical routine comes with many challenges. The algorithms should have high prediction accuracy but also be transparent, understandable and reliable. Thus, explainable artificial intelligence (XAI) is highly relevant for this domain. We present a survey on XAI within digital pathology, a medical imaging sub-discipline with particular characteristics and needs. The review includes several contributions. Firstly, we give a thorough overview of…
University of Toronto（多伦多大学）；Huron Digital Pathology Inc.（休伦数字病理公司）
One of the challenges facing the adoption of digital pathology workflows for clinical use is the need for automated quality control. As the scanners sometimes determine focus inaccurately, the resultant image blur deteriorates the scanned slide to the point of being unusable. Also, the scanned slide images tend to be extremely large when scanned at greater or equal 20X image resolution. Hence, for digital pathology to be clinically useful, it is necessary to use computational tools to quickly and accurately quantify the image focus quality and determine whether an image needs to be re-scanned.…
Explaining Digital Pathology Models via Clustering Activations
Faculty of Informatics, Masaryk University, Brno, Czech Republic（马萨里克大学信息学院，布拉格，捷克共和国）；Institute of Computer Science, Masaryk University, Brno, Czech Republic（马萨里克大学计算机科学研究所，布拉格，捷克共和国）；Masaryk Memorial Cancer Institute, Brno, Czech Republic（马萨里克纪念癌症研究所，布拉格，捷克共和国）
Machine learning (ML) algorithms are optimized for the distribution represented by the training data. For outlier data, they often deliver predictions with equal confidence, even though these should not be trusted. In order to deploy ML-based digital pathology solutions in clinical practice, effective methods for detecting anomalous data are crucial to avoid incorrect decisions in the outlier scenario. We propose a new unsupervised learning approach for anomaly detection in histopathology data based on generative adversarial networks (GANs). Compared to the existing GAN-based methods that have…
Discrepancy Minimization Improves Cross-Hospital Robustness in Digital Pathology
Reichman University, Herzliya, Israel（以色列海法大学）；Institute of Pathology, Sheba Medical Center, Ramat-Gan, Israel（以色列沙巴医疗中心病理研究所）；Technion - Israel Institute of Technology, Haifa, Israel（以色列技术学院）
Quantitative Benchmarking of Anomaly Detection Methods in Digital Pathology
Department of Computer Science Vanderbilt University, TN, USA（计算机科学系范德堡大学）；Department of Radiology Weill Cornell Medicine, NY, USA（放射学系韦尔·科恩医学中心）；Department of Medicine Vanderbilt University Medical Center, TN, USA Veterans Affairs Tennessee Valley Healthcare System TN, USA（医学系范德堡大学医学中心退伍军人事务田纳西山谷医疗系统）；Department of Medicine Vanderbilt University Medical Center Veterans Affairs Tennessee Valley Healthcare System TN, USA（医学系范德堡大学医学中心退伍军人事务田纳西山谷医疗系统）；Department of Anatomical Pathology UT MD Anderson Cancer Center,TX, USA（解剖病理学系UT MD安德森癌症中心）
In this work, we investigate multi-task learning as a way of pre-training models for classification tasks in digital pathology. It is motivated by the fact that many small and medium-size datasets have been released by the community over the years whereas there is no large scale dataset similar to ImageNet in the domain. We first assemble and transform many digital pathology datasets into a pool of 22 classification tasks and almost 900k images. Then, we propose a simple architecture and training scheme for creating a transferable model and a robust evaluation and selection protocol in order t…
Barco Healthcare（巴可医疗）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）
It is our conjecture that the variability of colors in a pathology image effects the interpretation of pathology cases, whether it is diagnostic accuracy, diagnostic confidence, or workflow efficiency. In this paper, digital pathology images are analyzed to quantify the perceived difference in color that occurs due to display aging, in particular a change in the maximum luminance, white point, and color gamut. The digital pathology images studied include diagnostically important features, such as the conspicuity of nuclei. Three different display aging models are applied to images: aging of lu…
University of Waterloo（滑铁卢大学）；Ontario Tech University（安大略理工大学）；Mayo Clinic（梅奥诊所）
Whole Slide Images (WSIs) in digital pathology are used to diagnose cancer subtypes. The difference in procedures to acquire WSIs at various trial sites gives rise to variability in the histopathology images, thus making consistent diagnosis challenging. These differences may stem from variability in image acquisition through multi-vendor scanners, variable acquisition parameters, and differences in staining procedure; as well, patient demographics may bias the glass slide batches before image acquisition. These variabilities are assumed to cause a domain shift in the images of different hospi…
University of Guelph（圭尔夫大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）
We consider annotation efficient learning in Digital Pathology (DP), where expert annotations are expensive and thus scarce. We explore the impact of image and input resolution on DP patch classification performance. We use two cancer patch classification datasets PCam and CRC, to validate the results of our study. Our experiments show that patch classification performance can be improved by manipulating both the image and input resolution in annotation-scarce and annotation-rich environments. We show a positive correlation between the image and input resolution and the patch classification ac…
IBM Research（IBM研究院）；EPFL（洛桑联邦理工学院）；ETH（苏黎世联邦理工学院）
Advances in entity-graph based analysis of histopathology images have brought in a new paradigm to describe tissue composition, and learn the tissue structure-to-function relationship. Entity-graphs offer flexible and scalable representations to characterize tissue organization, while allowing the incorporation of prior pathological knowledge to further support model interpretability and explainability. However, entity-graph analysis requires prerequisites for image-to-graph translation and knowledge of state-of-the-art machine learning algorithms applied to graph-structured data, which can po…
Normalizing unwanted color variations due to differences in staining processes and scanner responses has been shown to aid machine learning in computational pathology. Of the several popular techniques for color normalization, structure preserving color normalization (SPCN) is well-motivated, convincingly tested, and published with its code base. However, SPCN makes occasional errors in color basis estimation leading to artifacts such as swapping the color basis vectors between stains or giving a colored tinge to the background with no tissue. We made several algorithmic improvements to remove…
Jitendra Malik once said, "Supervision is the opium of the AI researcher". Most deep learning techniques heavily rely on extreme amounts of human labels to work effectively. In today's world, the rate of data creation greatly surpasses the rate of data annotation. Full reliance on human annotations is just a temporary means to solve current closed problems in AI. In reality, only a tiny fraction of data is annotated. Annotation Efficient Learning (AEL) is a study of algorithms to train models effectively with fewer annotations. To thrive in AEL environments, we need deep learning techniques th…
We propose a new model for digital pathology segmentation, based on the observation that histopathology images are inherently symmetric under rotation and reflection. Utilizing recent findings on rotation equivariant CNNs, the proposed model leverages these symmetries in a principled manner. We present a visual analysis showing improved stability on predictions, and demonstrate that exploiting rotation equivariance significantly improves tumor detection performance on a challenging lymph node metastases dataset. We further present a novel derived dataset to enable principled comparison of mach…
Sunnybrook Research Institute（桑尼布鲁克研究所）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）
To alleviate the burden of gathering detailed expert annotations when training deep neural networks, we propose a weakly supervised learning approach to recognize metastases in microscopic images of breast lymph nodes. We describe an alternative training loss which clusters weakly labeled bags in latent space to inform relevance of patch-instances during training of a convolutional neural network. We evaluate our method on the Camelyon dataset which contains high-resolution digital slides of breast lymph nodes, where labels are provided at the image-level and only subsets of patches are made a…
Unlocking adaptive digital pathology through dynamic feature learning
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；University of Oxford（牛津大学）；Fourth Military Medical University（第四军医大学）；Third Military Medical University（第三军医大学）；Chongqing Advanced Pathology Research Institute（重庆先进病理研究院）；Jinfeng Laboratory（金凤实验室）；Soochow University（苏州大学）；The Affiliated Cancer Hospital of Zhengzhou University and Henan Cancer Hospital（郑州大学附属肿瘤医院暨河南省肿瘤医院）；Research Institute of Tsinghua, Pearl River Delta（清华大学珠三角研究院）
DeepAf: One-Shot Spatiospectral Auto-Focus Model for Digital Pathology
Chair for Computer Aided Medical Procedures (CAMP), TU Munich, Germany（计算机辅助医学程序研究所（CAMP），慕尼黑技术大学，德国）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Southern Taiwan University of Science and Technology, Taiwan（台湾科技大学）；Harvard Data Science Initiative（哈佛大学数据科学计划）；Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA, USA（自然与人工智能研究学院，哈佛大学，剑桥，马萨诸塞州，美国）
Stony Brook University（石溪大学）；Brigham and Women's Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Fraunhofer Institute for Medical Image Computing MEVIS（弗劳恩霍夫医学图像计算研究所）；University of Bremen（不来梅大学）
Histopathological assessments, including surgical resection and core needle biopsy, are the standard procedures in the diagnosis of the prostate cancer. Current interpretation of the histopathology images includes the determination of the tumor area, Gleason grading, and identification of certain prognosis-critical features. Such a process is not only tedious, but also prune to intra/inter-observe variabilities. Recently, FDA cleared the marketing of the first whole slide imaging system for digital pathology. This opens a new era for the computer aided prostate image analysis and feature extra…
HistoEncoder: a digital pathology foundation model for prostate cancer
University of Helsinki（赫尔辛基大学）；Helsinki University Hospital（赫尔辛基大学医院）；Skåne University Hospital（斯科讷大学医院）；Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki（赫尔辛基大学HiLIFE项目下属芬兰分子医学研究所（FIMM））；iCAN Digital Precision Cancer Medicine Flagship（iCAN数字精准癌症医学旗舰项目）
Renal digital pathology visual knowledge search platform based on language large model and book knowledge
College of Information Technology, ZheJiang Shuren University（浙江树人大学信息技术学院）；The First Affiliated Hospital, School of Medicine, Zhejiang University（浙江大学医学院附属第一医院）；Chinese Academy of Medical Science（中国医学科学院）；Alibaba-Zhejiang University Joint Research Center of Future Digital Healthcare（阿里巴巴-浙江大学未来数字医疗联合研究中心）；Zhejiang University School of Medicine（浙江大学医学院）
CountPath: Automating Fragment Counting in Digital Pathology
Instituto Superior Técnico（高等技术学院）；INESC-ID；Faculdade de Engenharia da Universidade do Porto（波尔图大学工程学院）；INESC-TEC；IMP Diagnostics（IMP诊断公司）；Portuguese Oncology Institute of Porto（波尔图葡萄牙肿瘤研究所）
University of Science and Technology of China（中国科学技术大学）；UNSW Sydney（新南威尔士大学）；Genmab Inc.（Genmab公司）
NLP-based computer vision models, particularly vision transformers, have been shown to outperform CNN models in many imaging tasks. However, most digital pathology artificial-intelligence models are based on CNN architectures, probably owing to a lack of data regarding NLP models for pathology images. In this study, we developed digital pathology pipelines to benchmark the five most recently proposed NLP models (vision transformer (ViT), Swin Transformer, MobileViT, CMT, and Sequencer2D) and four popular CNN models (ResNet18, ResNet50, MobileNetV2, and EfficientNet) to predict biomarkers in co…
This project aims to break down large pathology images into small tiles and then cluster those tiles into distinct groups without the knowledge of true labels, our analysis shows how difficult certain aspects of clustering tumorous and non-tumorous cells can be and also shows that comparing the results of different unsupervised approaches is not a trivial task. The project also provides a software package to be used by the digital pathology community, that uses some of the approaches developed to perform unsupervised unsupervised tile classification, which could then be easily manually labelle…
University of Dayton（代顿大学）；Deep Lens Inc（Deep Lens公司）
Deep Learning (DL) approaches have been providing state-of-the-art performance in different modalities in the field of medical imagining including Digital Pathology Image Analysis (DPIA). Out of many different DL approaches, Deep Convolutional Neural Network (DCNN) technique provides superior performance for classification, segmentation, and detection tasks. Most of the task in DPIA problems are somehow possible to solve with classification, segmentation, and detection approaches. In addition, sometimes pre and post-processing methods are applied for solving some specific type of problems. Rec…
Comparing ImageNet Pre-training with Digital Pathology Foundation Models for Whole Slide Image-Based Survival Analysis
Imperial College London, Department of Electrical and Electronic Engineering, London, UK（帝国理工学院伦敦分校，电子与电气工程系）；Department of Pathology, Institut de Pathologie Multisite, Groupement Hospitalier Sud, Lyon University Hospital, Pierre-Bénite, France（病理学系，多站点病理研究所，南部医院集团，里昂大学医院，皮埃尔-贝内蒂，法国）；University of Lyon, Université Claude Bernard Lyon 1, Lyon, France（里昂大学， Claude Bernard 里昂第一大学，里昂，法国）
University of Guelph（圭尔夫大学）；Vector Institute（向量研究所）
A critical challenge of training deep learning models in the Digital Pathology (DP) domain is the high annotation cost by medical experts. One way to tackle this issue is via transfer learning from the natural image domain (NI), where the annotation cost is considerably cheaper. Cross-domain transfer learning from NI to DP is shown to be successful via class labels. One potential weakness of relying on class labels is the lack of spatial information, which can be obtained from spatial labels such as full pixel-wise segmentation labels and scribble labels. We demonstrate that scribble labels fr…
PatchSorter: A High Throughput Deep Learning Digital Pathology Tool for Object Labeling
Vetsuisse Faculty, University of Bern（伯尔尼大学兽医系）；Case Western Reserve University（凯斯西储大学）；Toth Technology LLC（托思科技有限责任公司）；Duke University（杜克大学）；Institute of Pathology, Lausanne University Hospital（洛桑大学医院病理研究所）；Bern Center for Precision Medicine, University of Bern（伯尔尼大学伯尔尼精准医学中心）；Emory University（埃默里大学）；Georgia Institute of Technology（佐治亚理工学院）；Atlanta Veterans Medical Center（亚特兰大退伍军人医疗中心）；The Netherlands Cancer Institute（荷兰癌症研究所）；University Hospital of Geneva（日内瓦大学医院）
本文发布开源数字病理标注工具 PatchSorter，通过结合深度学习与直观 Web 界面，在超过 100,000 个对象上实现较人工无辅助标注逾 7 倍的速度提升且几乎不损失准确性。
Siemens Healthineers（西门子医疗）；University Medical Center Goettingen（哥廷根大学医学中心）
Current approaches for classification of whole slide images (WSI) in digital pathology predominantly utilize a two-stage learning pipeline. The first stage identifies areas of interest (e.g. tumor tissue), while the second stage processes cropped tiles from these areas in a supervised fashion. During inference, a large number of tiles are combined into a unified prediction for the entire slide. A major drawback of such approaches is the requirement for task-specific auxiliary labels which are not acquired in clinical routine. We propose a novel learning pipeline for WSI classification that is …
