病理全玻片图像(WSI)的表示学习主要依赖于结合多示例学习(MIL)的弱监督方法。这种方法生成的玻片表示高度适配特定临床任务。自监督学习(SSL)已成功应用于训练组织病理基础模型(FM)以生成图像块嵌入。然而,生成患者级或玻片级嵌入仍然具有挑战性。现有的玻片表示学习方法通过对齐玻片的不同增强版本或利用多模态数据,将SSL的原理从图像块级别扩展到整张玻片。我们通过整合来自多个基础模型(FM)的图块嵌入,提出了一种在特征空间中生成有效玻片表示的单模态自监督学习新方法。我们的对比预训练策略名为COBRA,采用多个基础模型(FM)和基于Mamba-2的架构。尽管仅在来自The Cancer Genome Atlas(TCGA)的3048张全玻片图像(WSI)上进行预训练,COBRA在四个不同的公共Clinical Protemic Tumor Analysis Consortium(CPTAC)队列上的性能平均优于当前最优的玻片编码器至少4.4%的AUC。此外,COBRA在推理时可直接兼容此前未见过的特征提取器。代码可在https://github.com/KatherLab/COBRA获取。
英文摘要:
Representation learning of pathology whole-slide images (WSIs) has primarily relied on weak supervision with Multiple Instance Learning (MIL). This approach leads to slide representations highly tailored to a specific clinical task. Self-supervised learning (SSL) has been successfully applied to train histopathology foundation models (FMs) for patch embedding generation. However, generating patient or slide level embeddings remains challenging. Existing approaches for slide representation learning extend the principles of SSL from patch level learning to entire slides by aligning different augmentations of the slide or by utilizing multimodal data. By integrating tile embeddings from multiple FMs, we propose a new single modality SSL method in feature space that generates useful slide representations. Our contrastive pretraining strategy, called COBRA, employs multiple FMs and an architecture based on Mamba-2. COBRA exceeds performance of state-of-the-art slide encoders on four different public Clinical Protemic Tumor Analysis Consortium (CPTAC) cohorts on average by at least +4.4% AUC, despite only being pretrained on 3048 WSIs from The Cancer Genome Atlas (TCGA). Additionally, COBRA is readily compatible at inference time with previously unseen feature extractors. Code available at https://github.com/KatherLab/COBRA.
Shanghai Jiao Tong University(上海交通大学);ShanghaiTech University(上海科技大学);Shanghai United Imaging Intelligence Co., Ltd.(上海联影智能医疗科技有限公司);University of New South Wales(新南威尔士大学);BirenTech Research(壁仞科技研究院)
Can We Simplify Slide-level Fine-tuning of Pathology Foundation Models?
Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);University of Oxford(牛津大学);The First Affiliated Hospital of Sun Yat-sen University(中山大学附属第一医院);University of Washington(华盛顿大学)
Federated Distillation for Whole Slide Image via Gaussian-Mixture Feature Alignment and Curriculum Integration
School of Computer Science, Peking University, Beijing, China(北京大学计算机科学系);Center for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW 2113, Australia(健康信息学中心,澳大利亚健康创新研究所,麦考利大学,悉尼,NSW 2113,澳大利亚);School of Data Science, University of Virginia, Charlottesville, VA, USA(数据科学学院,弗吉尼亚大学,夏洛特维尔,VA,美国)
Faculty of Informatics and Data Science, University of Regensburg, Germany(信息学与数据科学学院,雷根萨大学);Indian Institute of Information Technology Bhopal, India(比哈尔信息学院)
From Whole-slide Image to Biomarker Prediction: A Protocol for End-to-End Deep Learning in Computational Pathology
Technical University Dresden(德累斯顿工业大学);University of St Andrews(圣安德鲁斯大学);Helmholtz Munich – German Research Center for Environment and Health(亥姆霍兹慕尼黑中心-德国环境与健康研究中心);Technical University of Munich(慕尼黑工业大学);University Hospital Aachen(亚琛大学医院);University Medical Center Mainz(美因茨大学医学中心);University of Leeds(利兹大学);National Center for Tumor Diseases (NCT)(国家肿瘤疾病中心 (NCT));University Hospital Heidelberg(海德堡大学医院);Technische Universität Dresden(德累斯顿工业大学)