通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2311.11772cs.CVcs.LG
用于全切片图像分类的病理特征提取器基准测试
Benchmarking Pathology Feature Extractors for Whole Slide Image Classification
University of St Andrews（圣安德鲁斯大学）；Else Kröner Fresenius Center for Digital Health（埃尔瑟·克勒纳·弗雷森纽斯数字健康中心）；TUD Dresden University of Technology（德累斯顿工业大学）；University Hospital Aachen（亚琛大学医院）；University Hospital Heidelberg（海德堡大学医院）；University Hospital Dresden（德累斯顿大学医院）
Georg Wölflein, Dyke Ferber, Asier R. Meneghetti, Omar S. M. El Nahhas, Daniel Truhn, Zunamys I. Carrero, David J. Harrison, Ognjen Arandjelović, Jakob Nikolas Kather
备注：
For the conference version see: arXiv:2311.11772v4. For the longer journal version with additional experiments see arXiv:2311.11772v5
Weakly supervised whole slide image classification is a key task in computational pathology, which involves predicting a slide-level label from a set of image patches constituting the slide. Constructing models to solve this task involves multiple design choices, often made without robust empirical or conclusive theoretical justification. To address this, we conduct a comprehensive benchmarking of feature extractors to answer three critical questions: 1) Is stain normalisation still a necessary preprocessing step? 2) Which feature extractors are best for downstream slide-level classification? 3) How does magnification affect downstream performance? Our study constitutes the most comprehensive evaluation of publicly available pathology feature extractors to date, involving more than 10,000 training runs across 14 feature extractors, 9 tasks, 5 datasets, 3 downstream architectures, 2 levels of magnification, and various preprocessing setups. Our findings challenge existing assumptions: 1) We observe empirically, and by analysing the latent space, that skipping stain normalisation and image augmentations does not degrade performance, while significantly reducing memory and computational demands. 2) We develop a novel evaluation metric to compare relative downstream performance, and show that the choice of feature extractor is the most consequential factor for downstream performance. 3) We find that lower-magnification slides are sufficient for accurate slide-level classification. Contrary to previous patch-level benchmarking studies, our approach emphasises clinical relevance by focusing on slide-level biomarker prediction tasks in a weakly supervised setting with external validation cohorts. Our findings stand to streamline digital pathology workflows by minimising preprocessing needs and informing the selection of feature extractors.
A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Model
The Hong Kong University of Science and Technology（香港科技大学）；Southern Medical University（南方医科大学）；Guangdong Provincial Key Laboratory of Molecular Tumor Pathology（广东省分子肿瘤病理学重点实验室）；Jinfeng Laboratory（金凤实验室）；Shandong First Medical University（山东第一医科大学）；Shandong Provincial Qianfoshan Hospital（山东省千佛山医院）；The Third Affiliated Hospital of Kunming Medical University（昆明医科大学第三附属医院）；Yunnan Cancer Hospital（云南省肿瘤医院）；The Chinese University of Hong Kong（香港中文大学）；Prince of Wales Hospital（威尔士亲王医院）；Sun Yat-sen University（中山大学）
ZeroSlide: Is Zero-Shot Classification Adequate for Lifelong Learning in Whole-Slide Image Analysis in the Era of Pathology Vision-Language Foundation Models?
Nara Institute of Science and Technology（奈良科学技术研究所）
Department of Pathology and Laboratory Medicine, University of Pennsylvania（病理学与实验室医学系，宾夕法尼亚大学）；Department of Biostatistics, Epidemiology & Informatics, University of Pennsylvania（生物统计学、流行病学与信息学系，宾夕法尼亚大学）；Department of Pathology, University of California at San Francisco（病理学系，加州大学旧金山分校）；Department of Pathology, Stanford University（病理学系，斯坦福大学）；Department of Electrical and System Engineering, University of Pennsylvania（电气与系统工程系，宾夕法尼亚大学）
Pretraining Multiple Instance Learning Networks with Multi-Teacher Distillation from Pathology Slide Foundation Models
Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；University of Washington（华盛顿大学）；City University of Hong Kong (Dongguan)（香港城市大学（东莞））；Medical Optical Technology R&D Center, Research Institute of Tsinghua（清华研究院医学光学技术研发中心）；Jinfeng Laboratory（金凤实验室）
Efficient Self-Supervised Barlow Twins from Limited Tissue Slide Cohorts for Colonic Pathology Diagnostics
University of Concordia（康考迪亚大学）；University of Montreal（蒙特利尔大学）；Sunnybrook Health Science Centre（桑尼布鲁克健康科学中心）；Kingston General Hospital（金斯顿总医院）；Mila–Quebec Artificial Intelligence Institute（米拉-魁北克人工智能研究所）
Single GPU Task Adaptation of Pathology Foundation Models for Whole Slide Image Analysis
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Icahn School of Medicine at Mount Sinai（辛格内医学中心）；Windreich Department of AI and Human Health（人工智能与人类健康部门）；Hasso Platner Institute at Mount Sinai（辛格内霍斯索普研究所）
本文提出单 GPU 病理基础模型任务适配方法 TAPFM，利用 ViT 注意力进行 MIL 聚合并分离计算图以稳定训练，在膀胱癌和肺腺癌突变预测上优于传统方法，使 PFM 适配在标准硬件上实用化。
Navigating Gigapixel Pathology Images with Large Multimodal Models
Department of Biomedical Informatics, Harvard Medical School（哈佛医学院生物医学信息学系）；Department of Pathology, Massachusetts General Hospital（麻省总医院病理学系）；Department of Pathology and Laboratory Medicine, Brown University（布朗大学病理学与实验室医学系）
MergeSurv: Merging-Based Continual Learning for Survival Analysis on Whole-Slide Images
University of Information Technology（信息技术大学）；Viet Nam National University Ho Chi Minh City（胡志明市越南国立大学）；ETIS (UMR 8051), CY Cergy Paris University, ENSEA, CNRS, France（法国CY Cergy巴黎大学、ENSEA、法国国家科学研究中心联合建立的ETIS实验室（UMR 8051））
Domain Adaptation Without the Compute Burden for Efficient Whole Slide Image Analysis
Institute for People-Centered AI, University of Surrey（以人为本的人工智能研究所，萨里大学）；Centre of Vision, Speech and Signal Processing, University of Surrey（视觉、语音和信号处理中心，萨里大学）
CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
College of Computer Science and Technology, Zhejiang University, China（浙江大学计算机科学与技术学院）；Research Center for Industries of the Future and School of Engineering, Westlake University, China（未来产业研究中心和西湖大学工程学院）；Department of Computer Science and Engineering, The Ohio State University, USA（俄亥俄州立大学计算机科学与工程系）
Benchmarking Pathology Foundation Models for Spatial Domain Understanding
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Brainnetome Center, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所脑网膜工程中心）；Beijing Key Laboratory of Brainnetome and Brain-Computer Interface, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所北京脑网膜与脑机接口重点实验室）；DAMO Academy, Alibaba Group（阿里云达摩院）；ShanghaiTech University（上海科技大学）
An interpretable machine learning system for colorectal cancer diagnosis from pathology slides
Institute for Systems and Computer Engineering, Technology and Science (INESC TEC)（系统与计算机工程、技术与科学研究所（INESC TEC））；Faculty of Engineering, University of Porto (FEUP)（波尔图大学工程学院）；IMP Diagnostics（IMP诊断公司）；Research Center of IPO Porto (CI-IPOP)（波尔图IPO癌症研究所研究中心（CI-IPOP））；Portuguese Oncology Institute of Porto (IPO Porto)（葡萄牙波尔图癌症研究所）；Porto Comprehensive Cancer Center (Porto.CCC)（波尔图综合癌症中心）；School of Medicine and Biomedical Sciences – University of Porto (ICBAS-UP)（波尔图大学医学与生物医学科学学院（ICBAS-UP））
AtlasPatch: Scalable Foundation Model-based Tissue Detection and Patch Extraction for Computational Pathology
Department of Computer Science and Software Engineering (CSSE)（计算机科学与软件工程系）；Concordia University（Concordia大学）；Concordia Institute for Information Systems Engineering (CIISE)（Concordia信息系统工程研究所）；Department of Building, Civil, and Environmental Engineering（建筑、土木和环境工程系）；University of Montreal Hospital Center (CHUM)（蒙特利尔大学医院中心）；Mila–Quebec AI Institute（Mila-魁北克人工智能研究所）；Khalifa University（Khalifa大学）；McGill University（麦吉尔大学）；Institute for Research in Immunology and Cancer（免疫学与癌症研究所在）
The application of deep learning to pathology assumes the existence of digital whole slide images of pathology slides. However, slide digitization is bottlenecked by the high cost of precise motor stages in slide scanners that are needed for position information used for slide stitching. We propose GloFlow, a two-stage method for creating a whole slide image using optical flow-based image registration with global alignment using a computationally tractable graph-pruning approach. In the first stage, we train an optical flow predictor to predict pairwise translations between successive video fr…
Bayer AG（拜耳股份公司）；Median Technologies（Median科技公司）；Bayer Crop Science（拜耳作物科学公司）；Pfizer Worldwide Research Development and Medical（辉瑞全球研发与医疗部门）
In pre-clinical pathology, there is a paradox between the abundance of raw data (whole slide images from many organs of many individual animals) and the lack of pixel-level slide annotations done by pathologists. Due to time constraints and requirements from regulatory authorities, diagnoses are instead stored as slide labels. Weakly supervised training is designed to take advantage of those data, and the trained models can be used by pathologists to rank slides by their probability of containing a given lesion of interest. In this work, we propose a novel contextualized eXplainable AI (XAI) f…
Paired Uterine Whole-Slide Images and Pathology Reports for Multimodal Computational Pathology
Institute of Pathology, Technical University of Munich（慕尼黑工业大学病理研究所）；Computer Aided Medical Procedures (CAMP), Technical University of Munich（慕尼黑工业大学计算机辅助医疗程序（CAMP））；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Department of Human Pathology, Juntendo University Graduate School of Medicine（顺天堂大学医学研究生院人体病理学部）；The Hong Kong University of Science and Technology（香港科技大学）；Munich Data Science Institute (MDSI)（慕尼黑数据科学研究所）
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Academy for Engineering and Technology, Fudan University（复旦大学工程与技术学院）；Fudan University Shanghai Cancer Center（复旦大学附属肿瘤医院）；Zhongshan Hospital, Fudan University（复旦大学中山医院）
Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Broad Institute of Harvard and MIT（哈佛-麻省理工学院博德研究所）；Dana-Farber Cancer Institute（达纳-法伯癌症研究所）
The rapidly emerging field of computational pathology has the potential to enable objective diagnosis, therapeutic response prediction and identification of new morphological features of clinical relevance. However, deep learning-based computational pathology approaches either require manual annotation of gigapixel whole slide images (WSIs) in fully-supervised settings or thousands of WSIs with slide-level labels in a weakly-supervised setting. Moreover, whole slide level computational pathology methods also suffer from domain adaptation and interpretability issues. These challenges have preve…
WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Images
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Research Center for Industries of the Future, Westlake University（西湖大学未来产业研究中心）；School of Engineering, Westlake University（西湖大学工程学院）
Pathology slides of lung malignancies are classified using resource-frugal convolution neural networks (CNNs) that may be deployed on mobile devices. In particular, the challenging task of distinguishing adenocarcinoma (LUAD) and squamous-cell carcinoma (LUSC) lung cancer subtypes is approached in two stages. First, whole-slide histopathology images are downsampled to a size too large for CNN analysis but large enough to retain key anatomic detail. The downsampled images are decomposed into smaller square tiles, which are sifted based on their image entropies. A lightweight CNN produces tile-l…
University of Toronto（多伦多大学）；Huron Digital Pathology Inc.（休伦数字病理公司）
One of the challenges facing the adoption of digital pathology workflows for clinical use is the need for automated quality control. As the scanners sometimes determine focus inaccurately, the resultant image blur deteriorates the scanned slide to the point of being unusable. Also, the scanned slide images tend to be extremely large when scanned at greater or equal 20X image resolution. Hence, for digital pathology to be clinically useful, it is necessary to use computational tools to quickly and accurately quantify the image focus quality and determine whether an image needs to be re-scanned.…
Memory-Augmented Incomplete Multimodal Survival Prediction via Cross-Slide and Gene-Attentive Hypergraph Learning
Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算机学院）；School of Software, Tsinghua University（清华大学软件学院）；School of Computer Science and Engineering, UNSW Sydney（新南威尔士大学计算机科学与工程学院）
This project aims to break down large pathology images into small tiles and then cluster those tiles into distinct groups without the knowledge of true labels, our analysis shows how difficult certain aspects of clustering tumorous and non-tumorous cells can be and also shows that comparing the results of different unsupervised approaches is not a trivial task. The project also provides a software package to be used by the digital pathology community, that uses some of the approaches developed to perform unsupervised unsupervised tile classification, which could then be easily manually labelle…
Brigham and Women’s Hospital（布里格姆妇女医院）；Harvard Medical School（哈佛医学院）；Broad Institute of Harvard and MIT（哈佛-麻省理工学院布罗德研究所）；Department of Pathology（病理系）；Northwell Health（诺斯韦尔健康）；Department of Biomedical Informatics（生物医学信息学系）；Dana-Farber Cancer Institute（达纳-法伯癌症研究所）；Cancer Program（癌症项目）；Cancer Data Science（癌症数据科学）
Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non-human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable, and accurate deep learning-based models often rely on the collection and time-costly curation large high-quality annotated training data that should ideally come from diverse sources and patient populations to cater for the heterogeneity that exists in such dataset…
Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception
College of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）
Can We Simplify Slide-level Fine-tuning of Pathology Foundation Models?
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；University of Oxford（牛津大学）；The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院）；University of Washington（华盛顿大学）
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）
In the clinic, resected tissue samples are stained with Hematoxylin-and-Eosin (H&E) and/or Immunhistochemistry (IHC) stains and presented to the pathologists on glass slides or as digital scans for diagnosis and assessment of disease progression. Cell-level quantification, e.g. in IHC protein expression scoring, can be extremely inefficient and subjective. We present DeepLIIF (https://deepliif.org), a first free online platform for efficient and reproducible IHC scoring. DeepLIIF outperforms current state-of-the-art approaches (relying on manual error-prone annotations) by virtually restaining…
Virchow: A Million-Slide Digital Pathology Foundation Model
Paige；Microsoft Research（微软研究院）；NSW Health Pathology（新南威尔士州卫生病理学）；St George Hospital（圣乔治医院）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；University of Rochester（罗切斯特大学）
ALICE: Learning a General-Purpose Pathology Foundation Model from Vision, Vision-Language, and Slide-Level Experts
Institute of Biopharmaceutical and Health Engineering, Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学生物医药与健康工程研究院，清华大学深圳国际研究生院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Department of Pathology, The First Affiliated Hospital of Sun Yat-sen University（中山大学附属第一医院病理科）；Medical Optical Technology R&D Center, Research Institute of Tsinghua, Pearl River Delta（清华珠三角研究院医学光学技术研发中心）
University of British Columbia（不列颠哥伦比亚大学）；Vancouver General Hospital（温哥华综合医院）；BC Cancer Research Center（不列颠哥伦比亚癌症研究中心）
Ovarian cancer is the most lethal cancer of the female reproductive organs. There are $5$ major histological subtypes of epithelial ovarian cancer, each with distinct morphological, genetic, and clinical features. Currently, these histotypes are determined by a pathologist's microscopic examination of tumor whole-slide images (WSI). This process has been hampered by poor inter-observer agreement (Cohen's kappa $0.54$-$0.67$). We utilized a \textit{two}-stage deep transfer learning algorithm based on convolutional neural networks (CNN) and progressive resizing for automatic classification of ep…
Geisel School of Medicine at Dartmouth（达特茅斯盖泽尔医学院）；Dartmouth-Hitchcock Medical Center（达特茅斯-希区柯克医疗中心）；Dartmouth College（达特茅斯学院）
In pathology, whole-slide images (WSI) based survival prediction has attracted increasing interest. However, given the large size of WSIs and the lack of pathologist annotations, extracting the prognostic information from WSIs remains a challenging task. Previous studies have used multiple instance learning approaches to combine the information from multiple randomly sampled patches, but different visual patterns may contribute differently to prognosis prediction. In this study, we developed a multi-head attention approach to focus on various parts of a tumor slide, for more comprehensive info…
