Mixed Magnification Aggregation for Generalizable Region-Level Representations in Computational Pathology
Microsoft Research（微软研究院）；Paige
Eric Zimmermann, Julian Viret, Michal Zelechowski, James Brian Hall, Neil Tenenholtz, Adam Casson, George Shaikovski, Eugene Vorontsov, Siqi Liu, Kristen A Severson
In recent years, a standard computational pathology workflow has emerged where whole slide images are cropped into tiles, these tiles are processed using a foundation model, and task-specific models are built using the resulting representations. At least 15 different foundation models have been proposed, and the vast majority are trained exclusively with tiles using the 20$\times$ magnification. However, it is well known that certain histologic features can only be discerned with larger context windows and requires a pathologist to zoom in and out when analyzing a whole slide image. Furthermore, creating 224$\times$224 pixel crops at 20$\times$ leads to a large number of tiles per slide, which can be gigapixel in size. To more accurately capture multi-resolution features and investigate the possibility of reducing the number of representations per slide, we propose a region-level mixing encoder. Our approach jointly fuses image tile representations of a mixed magnification foundation model using a masked embedding modeling pretraining step. We explore a design space for pretraining the proposed mixed-magnification region aggregators and evaluate our models on transfer to biomarker prediction tasks representing various cancer types. Results demonstrate cancer dependent improvements in predictive performance, highlighting the importance of spatial context and understanding.
Kyushu University（九州大学）；Kyoto University Hospital（京都大学医院）；National Institute of Informatics（国立信息学研究所）
Automated digital histopathology image segmentation is an important task to help pathologists diagnose tumors and cancer subtypes. For pathological diagnosis of cancer subtypes, pathologists usually change the magnification of whole-slide images (WSI) viewers. A key assumption is that the importance of the magnifications depends on the characteristics of the input image, such as cancer subtypes. In this paper, we propose a novel semantic segmentation method, called Adaptive-Weighting-Multi-Field-of-View-CNN (AWMF-CNN), that can adaptively use image features from images with different magnifica…
Mind the Gap: Continuous Magnification Sampling for Pathology Foundation Models
Berlin Institute for the Foundations of Learning and Data (BIFOLD)（柏林学习与数据基础研究所）；Machine Learning Group, Technische Universität Berlin（柏林技术大学机器学习小组）；Aignostics（Aignostics公司）；Charité, Universitätsmedizin Berlin（柏林夏里特大学医学院）
University of Waterloo（滑铁卢大学）；Huron Digital Pathology（休伦数字病理公司）
Recent advances in digital imaging has transformed computer vision and machine learning to new tools for analyzing pathology images. This trend could automate some of the tasks in the diagnostic pathology and elevate the pathologist workload. The final step of any cancer diagnosis procedure is performed by the expert pathologist. These experts use microscopes with high level of optical magnification to observe minute characteristics of the tissue acquired through biopsy and fixed on glass slides. Switching between different magnifications, and finding the magnification level at which they iden…
nnMIL: A generalizable multiple instance learning framework for computational pathology
Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, USA（放射肿瘤科，斯坦福大学医学院，斯坦福，CA，USA）；Stanford Institute for Human-Centered Artificial Intelligence, Stanford, CA, USA（斯坦福大学人本人工智能研究所，斯坦福，CA，USA）
In this work we show preliminary results of deep multi-task learning in the area of computational pathology. We combine 11 tasks ranging from patch-wise oral cancer classification, one of the most prevalent cancers in the developing world, to multi-tissue nuclei instance segmentation and classification.
A multi-modal vision-language model for generalizable annotation-free pathology localization
Paul C. Lauterbur Research Center for Biomedical Imaging（生物医学成像研究中心）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Pengcheng Laboratory（鹏城实验室）；University of Chinese Academy of Sciences（中国科学院大学）；Chinese Medicine Guangdong Laboratory（广东中医药实验室）；Beijing Chaoyang Hospital, Capital Medical University（首都医科大学北京朝阳医院）
本文提出无标注病理定位视觉-语言模型 AFLoc，通过多层次语义对比学习对齐医学概念与图像特征，在胸部 X 光及多模态数据上超越现有方法并展现强泛化能力。
Towards Computation- and Communication-efficient Computational Pathology
Guangdong Provincial Key Laboratory of Artificial Intelligence in Medical Image Analysis and Application（广东省人工智能在医学影像分析与应用重点实验室）；Guangdong Provincial People’s Hospital (Guangdong Academy of Medical Sciences)（广东省人民医院（广东省医学科学院））；Southern Medical University（南方医科大学）
EviPathBench: Benchmarking Evidence Acquisition and Reasoning in Vision-Language Models for Whole-Slide Pathology
National University of Singapore（新加坡国立大学）；PuzzleLogic Pte Ltd（拼图逻辑私人有限公司）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学深圳校区）；Peking Union Medical College Hospital（北京协和医院）
Toward Efficient Weakly Supervised Semantic Segmentation Using Only Low-Magnification Histopathological Images
University of Information Technology, Ho Chi Minh City, Vietnam（越南胡志明市信息科技学院）；Vietnam National University Ho Chi Minh City, Ho Chi Minh City, Vietnam（越南国家大学胡志明市分校）
Thinking in Scales: Accelerating Gigapixel Pathology Image Analysis via Adaptive Continuous Reasoning
School of Computer Science（计算机科学学院）；Technology, Xi’an Jiaotong University, Xi’an, China（技术学院，西安交通大学，西安，中国）；Department of Transmedia Art, Xi’an Academy of Fine Arts, Xi’an, China（多媒体艺术系，西安美术学院，西安，中国）；Department of Oncology, University of Cambridge, Cambridge, U.K.（肿瘤学系，剑桥大学，剑桥，英国）；Language Technology Lab, University of Cambridge, Cambridge, U.K.（语言技术实验室，剑桥大学，剑桥，英国）；Institute of High Performance Computing, Agency for Science, Technology（高性能计算研究所，科技研究局）
CORA: Generalizable coronary artery disease assessment and risk stratification from coronary CT angiography using pathology-centric representation learning
Department of Radiology, Northwestern University（西北大学放射学系）；Department of Biomedical Engineering, Northwestern University（西北大学生物医学工程系）；Department of Cardiology, Northwestern University（西北大学心脏病学系）；Department of Preventive Medicine, Northwestern University（西北大学预防医学系）
Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception
College of Computer Science and Technology, Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳）计算机科学与技术学院）；School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）
PathScale-R1: Cross-scale Reasoning for Pathological Image Analysis
National University of Singapore（新加坡国立大学）；PuzzleLogic Pte Ltd（拼图逻辑私人有限公司）；Fujian Medical University Cancer Hospital & Fujian Cancer Hospital（福建医科大学附属肿瘤医院）
Advancing H&E-to-IHC Stain Translation in Breast Cancer: A Multi-Magnification and Attention-Based Approach
Fudan University（复旦大学）；College of Electronic Engineering, Ocean University of China（中国海洋大学电子工程学院）；Research Center for Frontier Fundamental Studies, Zhejiang Lab（之江实验室前沿基础研究中心）；Institutes of Biomedical Sciences, Fudan University（复旦大学生物医学研究院）
University of Sydney（悉尼大学）；University of New South Wales（新南威尔士大学）；Microsoft Corporation（微软公司）；University of Pittsburgh（匹兹堡大学）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
We present a novel weakly-supervised framework for classifying whole slide images (WSIs). WSIs, due to their gigapixel resolution, are commonly processed by patch-wise classification with patch-level labels. However, patch-level labels require precise annotations, which is expensive and usually unavailable on clinical data. With image-level labels only, patch-wise classification would be sub-optimal due to inconsistency between the patch appearance and image-level label. To address this issue, we posit that WSI analysis can be effectively conducted by integrating information at both high magni…
Department of Pathology and Laboratory Medicine, University of Pennsylvania（病理学与实验室医学系，宾夕法尼亚大学）；Department of Biostatistics, Epidemiology & Informatics, University of Pennsylvania（生物统计学、流行病学与信息学系，宾夕法尼亚大学）；Department of Pathology, University of California at San Francisco（病理学系，加州大学旧金山分校）；Department of Pathology, Stanford University（病理学系，斯坦福大学）；Department of Electrical and System Engineering, University of Pennsylvania（电气与系统工程系，宾夕法尼亚大学）
School of Biomedical Engineering（生物医学工程学院）；Department of Pathology and Laboratory Medicine（病理与检验医学系）；Vancouver Prostate Centre（温哥华前列腺中心）；BC Cancer Institute（不列颠哥伦比亚省癌症研究所）；The University of British Columbia（不列颠哥伦比亚大学）
Benchmarking Pathology Feature Extractors for Whole Slide Image Classification
University of St Andrews（圣安德鲁斯大学）；Else Kröner Fresenius Center for Digital Health（埃尔瑟·克勒纳·弗雷森纽斯数字健康中心）；TUD Dresden University of Technology（德累斯顿工业大学）；University Hospital Aachen（亚琛大学医院）；University Hospital Heidelberg（海德堡大学医院）；University Hospital Dresden（德累斯顿大学医院）
Deep Learning-Based Fixation Type Prediction for Quality Assurance in Digital Pathology
Institute of Pathology, Technical University of Munich, Germany（慕尼黑技术大学病理学研究所）；School of Computation, Information and Technology, Technical University of Munich, Germany（慕尼黑技术大学计算、信息与技术学院）；Munich Center for Machine Learning, Germany（慕尼黑机器学习中心）；Institute of Pathology, University of Regensburg, Regensburg, Germany（罗森堡大学病理学研究所）；Institute of Pathology and Molecular Diagnostics, University Hospital of Augsburg, Germany（埃格利希大学医院病理学与分子诊断研究所）；Digital Medicine, University Hospital of Augsburg, Germany（埃格利希大学医院数字医学）；Bavarian Cancer Research Center (BZKF), Augsburg, Germany（巴伐利亚癌症研究中心（BZKF））；Bio M {}^{\text{M}} Biotech Cluster Development GmbH, Martinsried, Germany（Bio M {}^{\text{M}} 生物技术集群发展 GmbH）；Institute of Pathology, Medical Faculty Mannheim, University Heidelberg, Germany（海德堡大学曼海姆医学学院病理学研究所）；Munich Data Science Institute, Technical University of Munich, Germany（慕尼黑数据科学研究所）
Navigating Gigapixel Pathology Images with Large Multimodal Models
Department of Biomedical Informatics, Harvard Medical School（哈佛医学院生物医学信息学系）；Department of Pathology, Massachusetts General Hospital（麻省总医院病理学系）；Department of Pathology and Laboratory Medicine, Brown University（布朗大学病理学与实验室医学系）
A multimodal vision foundation model for generalizable knee pathology
Beihang University（北京航空航天大学）；Peking University Third Hospital（北京大学第三医院）；Institute of Sports Medicine of Peking University（北京大学运动医学研究所）；Ministry of Education（教育部）
Objective: We develop a computer-aided diagnosis (CAD) system using deep learning approaches for lesion detection and classification on whole-slide images (WSIs) with breast cancer. The deep features being distinguishing in classification from the convolutional neural networks (CNN) are demonstrated in this study to provide comprehensive interpretability for the proposed CAD system using pathological knowledge. Methods: In the experiment, a total of 186 slides of WSIs were collected and classified into three categories: Non-Carcinoma, Ductal Carcinoma in Situ (DCIS), and Invasive Ductal Carcin…
Towards A Generalizable Pathology Foundation Model via Unified Knowledge Distillation
The Hong Kong University of Science and Technology（香港科技大学）；Southern Medical University（南方医科大学）；Guangdong Provincial Key Laboratory of Molecular Tumor Pathology（广东省分子肿瘤病理学重点实验室）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The First Affiliated Hospital of Shandong First Medical University and Shandong Provincial Qianfoshan Hospital（山东第一医科大学第一附属医院暨山东省千佛山医院）；Sun Yat-sen University（中山大学）；The Chinese University of Hong Kong（香港中文大学）
Enhancing Pathological VLMs with Cross-scale Reasoning
Department of Electrical and Computer Engineering, National University of Singapore（新加坡国立大学电气与计算机工程系）；PuzzleLogic Pte Ltd（PuzzleLogic私人有限公司）；Department of Pathology, Fujian Medical University Cancer Hospital & Fujian Cancer Hospital（福建医科大学附属肿瘤医院病理科暨福建省肿瘤医院）
PathFL: Multi-Alignment Federated Learning for Pathology Image Segmentation
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education（新型一代人工智能技术及其交叉应用重点实验室（东南大学））；Department of Biostatistics, Center for Global Health, School of Public Health, Nanjing Medical University（公共卫生学院生物统计学系，全球健康中心，南京医科大学）；Institute of High-Performance Computing, Agency for Science, Technology and Research（高性能计算研究所，科技研究局）
DeepAf: One-Shot Spatiospectral Auto-Focus Model for Digital Pathology
Chair for Computer Aided Medical Procedures (CAMP), TU Munich, Germany（计算机辅助医学程序研究所（CAMP），慕尼黑技术大学，德国）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Southern Taiwan University of Science and Technology, Taiwan（台湾科技大学）；Harvard Data Science Initiative（哈佛大学数据科学计划）；Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University, Cambridge, MA, USA（自然与人工智能研究学院，哈佛大学，剑桥，马萨诸塞州，美国）
Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation
Department of Computer Science（计算机科学系）；ARIC；Khalifa University of Science and Technology（科技大学）；Information Technology University of the Punjab（旁遮普信息科技大学）；University of the Western Australia（西澳大学）
Central South University（中南大学）；Shaanxi Provincial People's Hospital（陕西省人民医院）
Background. Digital pathology has aroused widespread interest in modern pathology. The key of digitalization is to scan the whole slide image (WSI) at high magnification. The lager the magnification is, the richer details WSI will provide, but the scanning time is longer and the file size of obtained is larger. Methods. We design a strategy to scan slides with low resolution (5X) and a super-resolution method is proposed to restore the image details when in diagnosis. The method is based on a multi-scale generative adversarial network, which sequentially generates three high-resolution images …
MMAP: A Multi-Magnification and Prototype-Aware Architecture for Predicting Spatial Gene Expression
Institute for AI Innovation and Societal Impact（人工智能创新与社会影响研究所）；Hanoi University of Science and Technology（河内科学技术大学）；Amsterdam High School for the Gifted（阿姆斯特丹天才高中）；Anatomic Pathology Division, Laboratory Department, Vinmec Times City International Hospital（Vinmec国际医院解剖病理科实验室部门）；Vinmec Healthcare System（Vinmec医疗系统）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
CPathAgent: An Agent-based Foundation Model for Interpretable High-Resolution Pathology Image Analysis Mimicking Pathologists' Diagnostic Logic
College of Computer Science and Technology, Zhejiang University, China（浙江大学计算机科学与技术学院）；Research Center for Industries of the Future and School of Engineering, Westlake University, China（未来产业研究中心和西湖大学工程学院）；Department of Computer Science and Engineering, The Ohio State University, USA（俄亥俄州立大学计算机科学与工程系）
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Google Health（谷歌健康）
Metadata are general characteristics of the data in a well-curated and condensed format, and have been proven to be useful for decision making, knowledge discovery, and also heterogeneous data organization of biobank. Among all data types in the biobank, pathology is the key component of the biobank and also serves as the gold standard of diagnosis. To maximize the utility of biobank and allow the rapid progress of biomedical science, it is essential to organize the data with well-populated pathology metadata. However, manual annotation of such information is tedious and time-consuming. In the…
University of Warwick（华威大学）；Alan Turing Institute（艾伦·图灵研究所）；Institute for Infocomm Research（资讯通信研究院）；University of Sheffield（谢菲尔德大学）
While high-resolution pathology images lend themselves well to `data hungry' deep learning algorithms, obtaining exhaustive annotations on these images is a major challenge. In this paper, we propose a self-supervised CNN approach to leverage unlabeled data for learning generalizable and domain invariant representations in pathology images. The proposed approach, which we term as Self-Path, is a multi-task learning approach where the main task is tissue classification and pretext tasks are a variety of self-supervised tasks with labels inherent to the input data. We introduce novel domain spec…
AtlasPatch: Scalable Foundation Model-based Tissue Detection and Patch Extraction for Computational Pathology
Department of Computer Science and Software Engineering (CSSE)（计算机科学与软件工程系）；Concordia University（Concordia大学）；Concordia Institute for Information Systems Engineering (CIISE)（Concordia信息系统工程研究所）；Department of Building, Civil, and Environmental Engineering（建筑、土木和环境工程系）；University of Montreal Hospital Center (CHUM)（蒙特利尔大学医院中心）；Mila–Quebec AI Institute（Mila-魁北克人工智能研究所）；Khalifa University（Khalifa大学）；McGill University（麦吉尔大学）；Institute for Research in Immunology and Cancer（免疫学与癌症研究所在）
ETH Zurich（苏黎世联邦理工学院）；Uppsala University（乌普萨拉大学）；IBM Research Europe（IBM欧洲研究院）；Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）
Multiple Instance Learning (MIL) methods have become increasingly popular for classifying giga-pixel sized Whole-Slide Images (WSIs) in digital pathology. Most MIL methods operate at a single WSI magnification, by processing all the tissue patches. Such a formulation induces high computational requirements, and constrains the contextualization of the WSI-level representation to a single scale. A few MIL methods extend to multiple scales, but are computationally more demanding. In this paper, inspired by the pathological diagnostic process, we propose ZoomMIL, a method that learns to perform mu…
Suboptimal generalization of machine learning models on unseen data is a key challenge which hampers the clinical applicability of such models to medical imaging. Although various methods such as domain adaptation and domain generalization have evolved to combat this challenge, learning robust and generalizable representations is core to medical image understanding, and continues to be a problem. Here, we propose STRAP (Style TRansfer Augmentation for histoPathology), a form of data augmentation based on random style transfer from non-medical style source such as artistic paintings, for learni…
Enabling Progressive Whole-slide Image Analysis with Multi-scale Pyramidal Network
Institute for Regeneration and Repair, University of Edinburgh（再生与修复研究所，爱丁堡大学）；School of Informatics, University of Edinburgh（信息学院，爱丁堡大学）；Indica Labs, 8700 Education Pl NW, Bldg. B Albuquerque, US（Indica实验室，美国阿尔伯克基8700教育大道西北区B座）；Medical School, University of St Andrews（医学学校，圣安德鲁大学）
