arXiv 2405.102542024-05-22eess.IVcs.CVcs.LG

PRISM：用于切片级组织病理学的多模态生成基础模型

PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology

Paige（佩吉公司）；Microsoft Research（微软研究院）；University of Rochester（罗切斯特大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆-凯特琳癌症中心）

George Shaikovski, Adam Casson, Kristen Severson, Eric Zimmermann, Yi Kan Wang, Jeremy D. Kunz, Juan A. Retamero, Gerard Oakley, David Klimstra, Christopher Kan… 展开作者

George Shaikovski, Adam Casson, Kristen Severson, Eric Zimmermann, Yi Kan Wang, Jeremy D. Kunz, Juan A. Retamero, Gerard Oakley, David Klimstra, Christopher Kanan, Matthew Hanna, Michal Zelechowski, Julian Viret, Neil Tenenholtz, James Hall, Nicolo Fusi, Razik Yousfi, Peter Hamilton, William A. Moye, Eugene Vorontsov, Siqi Liu, Thomas J. Fuchs

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

PRISM是一个基于Virchow嵌入和临床报告文本预训练的切片级组织病理学基础模型，通过文本提示实现零样本癌症检测，并在微调后以少量数据超越监督基线。

摘要：

计算病理学中的基础模型有望推动新的临床决策支持系统和精准医学模型的发展。然而，大多数临床分析是在一张或多张全切片图像的层面定义的，而迄今为止的基础模型则分别处理全切片图像中包含的数千个图像块，这两者之间存在不匹配。训练一个网络来聚合多张全切片图像中大量图像块的信息这一需求限制了这些模型的影响力。在这项工作中，我们提出了一个用于H&E染色的组织病理学的切片级基础模型PRISM，它基于Virchow图像块嵌入，并利用临床报告文本进行预训练。利用图像块嵌入，PRISM生成切片级嵌入，并具有生成临床报告的能力，从而产生多种使用模式。通过文本提示，PRISM实现了零样本癌症检测和亚型分类，其性能接近并超过监督聚合器模型。使用线性分类器对切片嵌入进行分类时，PRISM超过了监督聚合器模型。此外，我们证明了PRISM切片编码器的微调能够为生物标志物预测提供标签高效的训练，而这一任务通常面临训练数据可用性低的问题；用PRISM初始化的聚合器仅使用10%的训练数据进行训练，就能胜过使用全部数据的监督基线。

英文摘要：

Foundation models in computational pathology promise to unlock the development of new clinical decision support systems and models for precision medicine. However, there is a mismatch between most clinical analysis, which is defined at the level of one or more whole slide images, and foundation models to date, which process the thousands of image tiles contained in a whole slide image separately. The requirement to train a network to aggregate information across a large number of tiles in multiple whole slide images limits these models' impact. In this work, we present a slide-level foundation model for H&E-stained histopathology, PRISM, that builds on Virchow tile embeddings and leverages clinical report text for pre-training. Using the tile embeddings, PRISM produces slide-level embeddings with the ability to generate clinical reports, resulting in several modes of use. Using text prompts, PRISM achieves zero-shot cancer detection and sub-typing performance approaching and surpassing that of a supervised aggregator model. Using the slide embeddings with linear classifiers, PRISM surpasses supervised aggregator models. Furthermore, we demonstrate that fine-tuning of the PRISM slide encoder yields label-efficient training for biomarker prediction, a task that typically suffers from low availability of training data; an aggregator initialized with PRISM and trained on as little as 10% of the training data can outperform a supervised baseline that uses all of the data.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。