Whole slide images~(WSIs) are digitized images of tissues placed in glass slides using advanced scanners. The digital processing of WSIs is challenging as they are gigapixel images and stored in multi-resolution format. A common challenge with WSIs is that perturbations/artifacts are inevitable during storing the glass slides and digitizing them. These perturbations include motion, which often arises from slide movement during placement, and changes in hue and brightness due to variations in staining chemicals and the quality of digitizing scanners. In this work, a novel robust learning approach to account for these artifacts is presented. Due to the size and resolution of WSIs and to account for neighborhood information, graph-based methods are called for. We use graph convolutional network~(GCN) to extract features from the graph representing WSI. Through a denoiser {and pooling layer}, the effects of perturbations in WSIs are controlled and the output is followed by a transformer for the classification of different grades of prostate cancer. To compare the efficacy of the proposed approach, the model without denoiser is trained and tested with WSIs without any perturbation and then different perturbations are introduced in WSIs and passed through the network with the denoiser. The accuracy and kappa scores of the proposed model with prostate cancer dataset compared with non-robust algorithms show significant improvement in cancer diagnosis.
University of Toronto（多伦多大学）；Huron Digital Pathology Inc.（休伦数字病理公司）
One of the challenges facing the adoption of digital pathology workflows for clinical use is the need for automated quality control. As the scanners sometimes determine focus inaccurately, the resultant image blur deteriorates the scanned slide to the point of being unusable. Also, the scanned slide images tend to be extremely large when scanned at greater or equal 20X image resolution. Hence, for digital pathology to be clinically useful, it is necessary to use computational tools to quickly and accurately quantify the image focus quality and determine whether an image needs to be re-scanned.…
Quantitative Benchmarking of Anomaly Detection Methods in Digital Pathology
Department of Computer Science Vanderbilt University, TN, USA（计算机科学系范德堡大学）；Department of Radiology Weill Cornell Medicine, NY, USA（放射学系韦尔·科恩医学中心）；Department of Medicine Vanderbilt University Medical Center, TN, USA Veterans Affairs Tennessee Valley Healthcare System TN, USA（医学系范德堡大学医学中心退伍军人事务田纳西山谷医疗系统）；Department of Medicine Vanderbilt University Medical Center Veterans Affairs Tennessee Valley Healthcare System TN, USA（医学系范德堡大学医学中心退伍军人事务田纳西山谷医疗系统）；Department of Anatomical Pathology UT MD Anderson Cancer Center,TX, USA（解剖病理学系UT MD安德森癌症中心）
IBM Research（IBM研究院）；EPFL（洛桑联邦理工学院）；ETH（苏黎世联邦理工学院）
Advances in entity-graph based analysis of histopathology images have brought in a new paradigm to describe tissue composition, and learn the tissue structure-to-function relationship. Entity-graphs offer flexible and scalable representations to characterize tissue organization, while allowing the incorporation of prior pathological knowledge to further support model interpretability and explainability. However, entity-graph analysis requires prerequisites for image-to-graph translation and knowledge of state-of-the-art machine learning algorithms applied to graph-structured data, which can po…
Unlocking adaptive digital pathology through dynamic feature learning
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；University of Oxford（牛津大学）；Fourth Military Medical University（第四军医大学）；Third Military Medical University（第三军医大学）；Chongqing Advanced Pathology Research Institute（重庆先进病理研究院）；Jinfeng Laboratory（金凤实验室）；Soochow University（苏州大学）；The Affiliated Cancer Hospital of Zhengzhou University and Henan Cancer Hospital（郑州大学附属肿瘤医院暨河南省肿瘤医院）；Research Institute of Tsinghua, Pearl River Delta（清华大学珠三角研究院）
Comparing ImageNet Pre-training with Digital Pathology Foundation Models for Whole Slide Image-Based Survival Analysis
Imperial College London, Department of Electrical and Electronic Engineering, London, UK（帝国理工学院伦敦分校，电子与电气工程系）；Department of Pathology, Institut de Pathologie Multisite, Groupement Hospitalier Sud, Lyon University Hospital, Pierre-Bénite, France（病理学系，多站点病理研究所，南部医院集团，里昂大学医院，皮埃尔-贝内蒂，法国）；University of Lyon, Université Claude Bernard Lyon 1, Lyon, France（里昂大学， Claude Bernard 里昂第一大学，里昂，法国）
Linköping University（林雪平大学）；Center for Medical Image Science and Visualization, Linköping University（林雪平大学医学影像科学与可视化中心）；Sectra AB
Artificial intelligence (AI) has shown great promise for diagnostic imaging assessments. However, the application of AI to support medical diagnostics in clinical routine comes with many challenges. The algorithms should have high prediction accuracy but also be transparent, understandable and reliable. Thus, explainable artificial intelligence (XAI) is highly relevant for this domain. We present a survey on XAI within digital pathology, a medical imaging sub-discipline with particular characteristics and needs. The review includes several contributions. Firstly, we give a thorough overview of…
Machine learning (ML) algorithms are optimized for the distribution represented by the training data. For outlier data, they often deliver predictions with equal confidence, even though these should not be trusted. In order to deploy ML-based digital pathology solutions in clinical practice, effective methods for detecting anomalous data are crucial to avoid incorrect decisions in the outlier scenario. We propose a new unsupervised learning approach for anomaly detection in histopathology data based on generative adversarial networks (GANs). Compared to the existing GAN-based methods that have…
In this work, we investigate multi-task learning as a way of pre-training models for classification tasks in digital pathology. It is motivated by the fact that many small and medium-size datasets have been released by the community over the years whereas there is no large scale dataset similar to ImageNet in the domain. We first assemble and transform many digital pathology datasets into a pool of 22 classification tasks and almost 900k images. Then, we propose a simple architecture and training scheme for creating a transferable model and a robust evaluation and selection protocol in order t…
SagivTech Ltd.（SagivTech有限公司）；Roche Innovation Center Munich（慕尼黑罗氏创新中心）
Using features extracted from networks pretrained on ImageNet is a common practice in applications of deep learning for digital pathology. However it presents the downside of missing domain specific image information. In digital pathology, supervised training data is expensive and difficult to collect. We propose a self-supervised method for feature extraction by similarity learning on whole slide images (WSI) that is simple to implement and allows creation of robust and compact image descriptors. We train a siamese network, exploiting image spatial continuity and assuming spatially adjacent t…
Unlocking the Potential of Digital Pathology: Novel Baselines for Compression
German Cancer Research Center (DKFZ)（德国癌症研究中心 (DKFZ)）；Heidelberg University（海德堡大学）；Ulm University Medical Centre（乌尔姆大学医学中心）；Technical University of Munich（慕尼黑工业大学）；University Medicine Essen（埃森大学医学中心）；Heidelberg University Hospital（海德堡大学医院）；PixelMed Publishing
Leveraging Weak Supervision for Cell Localization in Digital Pathology Using Multitask Learning and Consistency Loss
Koc University（科克大学）；Basaksehir Cam and Sakura City Hospital（巴沙克谢伊尔卡姆与樱花城市医院）；Sisli Hamidiye Etfal Health Application and Research Center（锡什利哈米迪耶埃特法尔健康应用与研究中心）；Koc University School of Medicine（科克大学医学院）
Kharkevich Institute for Information Transmission Problems（信息传输问题研究所（Kharkevich研究所））；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
In the last years, neural networks have proven to be a powerful framework for various image analysis problems. However, some application domains have specific limitations. Notably, digital pathology is an example of such fields due to tremendous image sizes and quite limited number of training examples available. In this paper, we adopt state-of-the-art convolutional neural networks (CNN) architectures for digital pathology images analysis. We propose to classify image patches to increase effective sample size and then to apply an ensembling technique to build prediction for the original image…
University of Guelph（圭尔夫大学）；Vector Institute（向量研究所）
In Digital Pathology (DP), labeled data is generally very scarce due to the requirement that medical experts provide annotations. We address this issue by learning transferable features from weakly labeled data, which are collected from various parts of the body and are organized by non-medical experts. In this paper, we show that features learned from such weakly labeled datasets are indeed transferable and allow us to achieve highly competitive patch classification results on the colorectal cancer (CRC) dataset [1] and the PatchCamelyon (PCam) dataset [2] while using an order of magnitude le…
Renal digital pathology visual knowledge search platform based on language large model and book knowledge
College of Information Technology, ZheJiang Shuren University（浙江树人大学信息技术学院）；The First Affiliated Hospital, School of Medicine, Zhejiang University（浙江大学医学院附属第一医院）；Chinese Academy of Medical Science（中国医学科学院）；Alibaba-Zhejiang University Joint Research Center of Future Digital Healthcare（阿里巴巴-浙江大学未来数字医疗联合研究中心）；Zhejiang University School of Medicine（浙江大学医学院）
MICS Laboratory, CentraleSupélec, Université Paris-Saclay（巴黎萨克雷大学中央理工高等电力学院MICS实验室）；Department of Radiotherapy, Gustave Roussy Cancer Campus（古斯塔夫·鲁西癌症园区放射治疗科）；Therapanacea（Therapanacea公司）
The examination of histopathology images is considered to be the gold standard for the diagnosis and stratification of cancer patients. A key challenge in the analysis of such images is their size, which can run into the gigapixels and can require tedious screening by clinicians. With the recent advances in computational medicine, automatic tools have been proposed to assist clinicians in their everyday practice. Such tools typically process these large images by slicing them into tiles that can then be encoded and utilized for different clinical models. In this study, we propose a novel gener…
Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are…
This project aims to break down large pathology images into small tiles and then cluster those tiles into distinct groups without the knowledge of true labels, our analysis shows how difficult certain aspects of clustering tumorous and non-tumorous cells can be and also shows that comparing the results of different unsupervised approaches is not a trivial task. The project also provides a software package to be used by the digital pathology community, that uses some of the approaches developed to perform unsupervised unsupervised tile classification, which could then be easily manually labelle…
University of Guelph（圭尔夫大学）；Vector Institute（向量研究所）
A critical challenge of training deep learning models in the Digital Pathology (DP) domain is the high annotation cost by medical experts. One way to tackle this issue is via transfer learning from the natural image domain (NI), where the annotation cost is considerably cheaper. Cross-domain transfer learning from NI to DP is shown to be successful via class labels. One potential weakness of relying on class labels is the lack of spatial information, which can be obtained from spatial labels such as full pixel-wise segmentation labels and scribble labels. We demonstrate that scribble labels fr…
Siemens Healthineers（西门子医疗）；University Medical Center Goettingen（哥廷根大学医学中心）
Current approaches for classification of whole slide images (WSI) in digital pathology predominantly utilize a two-stage learning pipeline. The first stage identifies areas of interest (e.g. tumor tissue), while the second stage processes cropped tiles from these areas in a supervised fashion. During inference, a large number of tiles are combined into a unified prediction for the entire slide. A major drawback of such approaches is the requirement for task-specific auxiliary labels which are not acquired in clinical routine. We propose a novel learning pipeline for WSI classification that is …
High-performance Data Management for Whole Slide Image Analysis in Digital Pathology
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）；NVIDIA Corporation（英伟达公司）；Georgia State University（佐治亚州立大学）；Oak Ridge National Laboratory（橡树岭国家实验室）；Weill Cornell Medicine（威尔康奈尔医学院）
本文提出基于 ADIOS2 的数字病理 WSI 数据管理流水线，通过并行化补丁级 I/O 缩短检索时间，在 CPU 场景获两倍加速，并在 GPU 场景达到与 NVIDIA Magnum IO GDS 相当的性能。
EPFL（洛桑联邦理工学院）；IBM Research Zürich（IBM苏黎世研究院）；ETH Zürich（苏黎世联邦理工学院）；National Cancer Institute - IRCCS-Fondazione Pascale（国家癌症研究所 - IRCCS帕斯卡尔基金会）
Explainability of machine learning (ML) techniques in digital pathology (DP) is of great significance to facilitate their wide adoption in clinics. Recently, graph techniques encoding relevant biological entities have been employed to represent and assess DP images. Such paradigm shift from pixel-wise to entity-wise analysis provides more control over concept representation. In this paper, we introduce a post-hoc explainer to derive compact per-instance explanations emphasizing diagnostically important entities in the graph. Although we focus our analyses to cells and cellular interactions in …
University of Science and Technology of China（中国科学技术大学）；UNSW Sydney（新南威尔士大学）；Genmab Inc.（Genmab公司）
NLP-based computer vision models, particularly vision transformers, have been shown to outperform CNN models in many imaging tasks. However, most digital pathology artificial-intelligence models are based on CNN architectures, probably owing to a lack of data regarding NLP models for pathology images. In this study, we developed digital pathology pipelines to benchmark the five most recently proposed NLP models (vision transformer (ViT), Swin Transformer, MobileViT, CMT, and Sequencer2D) and four popular CNN models (ResNet18, ResNet50, MobileNetV2, and EfficientNet) to predict biomarkers in co…
Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs
University of Stavanger（斯塔万格大学）；University Medical Center Rotterdam（鹿特丹大学医学中心）；Erasmus MC Cancer Institute（伊拉斯姆斯大学医学中心癌症研究所）；Stavanger University Hospital（斯塔万格大学医院）；INCLIVA Biomedical Research Institute（INCLIVA生物医学研究所）；University of Valencia（瓦伦西亚大学）
Australian Breast Cancer Tissue Bank, University of Sydney（澳大利亚乳腺癌症组织库，悉尼大学）；BioMed Central Ltd（BioMed Central有限公司）
Virtual microscopy includes digitisation of histology slides and the use of computer technologies for complex investigation of diseases such as cancer. However, automated image analysis, or website publishing of such digital images, is hampered by their large file sizes. We have developed two Java based open source tools: Snapshot Creator and NDPI-Splitter. Snapshot Creator converts a portion of a large digital slide into a desired quality JPEG image. The image is linked to the patients clinical and treatment information in a customised open source cancer data management software (Caisis) in u…
Explaining Digital Pathology Models via Clustering Activations
Faculty of Informatics, Masaryk University, Brno, Czech Republic（马萨里克大学信息学院，布拉格，捷克共和国）；Institute of Computer Science, Masaryk University, Brno, Czech Republic（马萨里克大学计算机科学研究所，布拉格，捷克共和国）；Masaryk Memorial Cancer Institute, Brno, Czech Republic（马萨里克纪念癌症研究所，布拉格，捷克共和国）
Assessing and Enhancing Robustness of Deep Learning Models with Corruption Emulation in Digital Pathology
College of Engineering, Peking University（北京大学工学院）；Third Hospital, Peking University Health Science Center（北京大学医学部第三医院）；Institute of Biomedical Engineering, Beijing Institute of Collaborative Innovation（北京协同创新研究院生物医学工程研究所）
Discrepancy Minimization Improves Cross-Hospital Robustness in Digital Pathology
Reichman University, Herzliya, Israel（以色列海法大学）；Institute of Pathology, Sheba Medical Center, Ramat-Gan, Israel（以色列沙巴医疗中心病理研究所）；Technion - Israel Institute of Technology, Haifa, Israel（以色列技术学院）
Barco Healthcare（巴可医疗）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）
It is our conjecture that the variability of colors in a pathology image effects the interpretation of pathology cases, whether it is diagnostic accuracy, diagnostic confidence, or workflow efficiency. In this paper, digital pathology images are analyzed to quantify the perceived difference in color that occurs due to display aging, in particular a change in the maximum luminance, white point, and color gamut. The digital pathology images studied include diagnostically important features, such as the conspicuity of nuclei. Three different display aging models are applied to images: aging of lu…
University of Waterloo（滑铁卢大学）；Ontario Tech University（安大略理工大学）；Mayo Clinic（梅奥诊所）
Whole Slide Images (WSIs) in digital pathology are used to diagnose cancer subtypes. The difference in procedures to acquire WSIs at various trial sites gives rise to variability in the histopathology images, thus making consistent diagnosis challenging. These differences may stem from variability in image acquisition through multi-vendor scanners, variable acquisition parameters, and differences in staining procedure; as well, patient demographics may bias the glass slide batches before image acquisition. These variabilities are assumed to cause a domain shift in the images of different hospi…
University of Guelph（圭尔夫大学）；Vector Institute for Artificial Intelligence（矢量人工智能研究所）
We consider annotation efficient learning in Digital Pathology (DP), where expert annotations are expensive and thus scarce. We explore the impact of image and input resolution on DP patch classification performance. We use two cancer patch classification datasets PCam and CRC, to validate the results of our study. Our experiments show that patch classification performance can be improved by manipulating both the image and input resolution in annotation-scarce and annotation-rich environments. We show a positive correlation between the image and input resolution and the patch classification ac…
Normalizing unwanted color variations due to differences in staining processes and scanner responses has been shown to aid machine learning in computational pathology. Of the several popular techniques for color normalization, structure preserving color normalization (SPCN) is well-motivated, convincingly tested, and published with its code base. However, SPCN makes occasional errors in color basis estimation leading to artifacts such as swapping the color basis vectors between stains or giving a colored tinge to the background with no tissue. We made several algorithmic improvements to remove…
Stain Consistency Learning: Handling Stain Variation for Automatic Digital Pathology Segmentation
Imperial College London（伦敦帝国理工学院）；The Babraham Institute（巴布拉汉研究所）；University of Cambridge（剑桥大学）；National Heart & Lung Institute, Imperial College London（伦敦帝国理工学院国家心肺研究所）；Royal Brompton Hospital（皇家布朗普顿医院）
Jitendra Malik once said, "Supervision is the opium of the AI researcher". Most deep learning techniques heavily rely on extreme amounts of human labels to work effectively. In today's world, the rate of data creation greatly surpasses the rate of data annotation. Full reliance on human annotations is just a temporary means to solve current closed problems in AI. In reality, only a tiny fraction of data is annotated. Annotation Efficient Learning (AEL) is a study of algorithms to train models effectively with fewer annotations. To thrive in AEL environments, we need deep learning techniques th…
We propose a new model for digital pathology segmentation, based on the observation that histopathology images are inherently symmetric under rotation and reflection. Utilizing recent findings on rotation equivariant CNNs, the proposed model leverages these symmetries in a principled manner. We present a visual analysis showing improved stability on predictions, and demonstrate that exploiting rotation equivariance significantly improves tumor detection performance on a challenging lymph node metastases dataset. We further present a novel derived dataset to enable principled comparison of mach…
Sunnybrook Research Institute（桑尼布鲁克研究所）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）
To alleviate the burden of gathering detailed expert annotations when training deep neural networks, we propose a weakly supervised learning approach to recognize metastases in microscopic images of breast lymph nodes. We describe an alternative training loss which clusters weakly labeled bags in latent space to inform relevance of patch-instances during training of a convolutional neural network. We evaluate our method on the Camelyon dataset which contains high-resolution digital slides of breast lymph nodes, where labels are provided at the image-level and only subsets of patches are made a…
A robust and scalable framework for hallucination detection in virtual tissue staining and digital pathology
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；California NanoSystems Institute (CNSI)（加州纳米系统研究所）；Hadassah Hebrew University Medical Center（哈达萨希伯来大学医学中心）；University of Southern California（南加利福尼亚大学）
HistoColAi: An Open-Source Web Platform for Collaborative Digital Histology Image Annotation with AI-Driven Predictive Integration
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València（瓦伦西亚理工大学以人为中心技术大学研究院）；valgrAI: Valencian Graduate School and Research Network of Artificial Intelligence（瓦伦西亚人工智能研究生院及研究网络）；ÉTS Montréal（蒙特利尔工程学院）
A Spatially-Aware Multiple Instance Learning Framework for Digital Pathology
Department of Biomedical Engineering, Eindhoven University of Technology（生物医学工程系，埃因霍温理工大学）；Mathematics and Computer Science Department, Eindhoven University of Technology（数学与计算机科学系，埃因霍温理工大学）
Stony Brook University（石溪大学）；Brigham and Women's Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Fraunhofer Institute for Medical Image Computing MEVIS（弗劳恩霍夫医学图像计算研究所）；University of Bremen（不来梅大学）
Histopathological assessments, including surgical resection and core needle biopsy, are the standard procedures in the diagnosis of the prostate cancer. Current interpretation of the histopathology images includes the determination of the tumor area, Gleason grading, and identification of certain prognosis-critical features. Such a process is not only tedious, but also prune to intra/inter-observe variabilities. Recently, FDA cleared the marketing of the first whole slide imaging system for digital pathology. This opens a new era for the computer aided prostate image analysis and feature extra…
