arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2411.13623cs.CV

无监督的基础模型无关的玻片级表示学习

Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning

EKFZ for Digital Health TU Dresden(德累斯顿工业大学EKFZ数字健康中心);University of St Andrews(圣安德鲁斯大学);Heidelberg University(海德堡大学)

Tim Lenz, Peter Neidlinger, Marta Ligero, Georg Wölflein, Marko van Treeck, Jakob Nikolas Kather

AI总结:

针对病理全玻片图像的玻片级表示学习难题,提出基于多基础模型与Mamba-2架构的单模态自监督对比预训练策略COBRA,在小样本预训练下实现性能提升且兼容新特征提取器。

摘要:

病理全玻片图像(WSI)的表示学习主要依赖于结合多示例学习(MIL)的弱监督方法。这种方法生成的玻片表示高度适配特定临床任务。自监督学习(SSL)已成功应用于训练组织病理基础模型(FM)以生成图像块嵌入。然而,生成患者级或玻片级嵌入仍然具有挑战性。现有的玻片表示学习方法通过对齐玻片的不同增强版本或利用多模态数据,将SSL的原理从图像块级别扩展到整张玻片。我们通过整合来自多个基础模型(FM)的图块嵌入,提出了一种在特征空间中生成有效玻片表示的单模态自监督学习新方法。我们的对比预训练策略名为COBRA,采用多个基础模型(FM)和基于Mamba-2的架构。尽管仅在来自The Cancer Genome Atlas(TCGA)的3048张全玻片图像(WSI)上进行预训练,COBRA在四个不同的公共Clinical Protemic Tumor Analysis Consortium(CPTAC)队列上的性能平均优于当前最优的玻片编码器至少4.4%的AUC。此外,COBRA在推理时可直接兼容此前未见过的特征提取器。代码可在https://github.com/KatherLab/COBRA获取。

英文摘要:

Representation learning of pathology whole-slide images (WSIs) has primarily relied on weak supervision with Multiple Instance Learning (MIL). This approach leads to slide representations highly tailored to a specific clinical task. Self-supervised learning (SSL) has been successfully applied to train histopathology foundation models (FMs) for patch embedding generation. However, generating patient or slide level embeddings remains challenging. Existing approaches for slide representation learning extend the principles of SSL from patch level learning to entire slides by aligning different augmentations of the slide or by utilizing multimodal data. By integrating tile embeddings from multiple FMs, we propose a new single modality SSL method in feature space that generates useful slide representations. Our contrastive pretraining strategy, called COBRA, employs multiple FMs and an architecture based on Mamba-2. COBRA exceeds performance of state-of-the-art slide encoders on four different public Clinical Protemic Tumor Analysis Consortium (CPTAC) cohorts on average by at least +4.4% AUC, despite only being pretrained on 3048 WSIs from The Cancer Genome Atlas (TCGA). Additionally, COBRA is readily compatible at inference time with previously unseen feature extractors. Code available at https://github.com/KatherLab/COBRA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑