arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2506.13063cs.CVcs.CL

PRISM2:通过临床对话解锁多模态通用病理学AI

PRISM2: Unlocking Multi-Modal General Pathology AI with Clinical Dialogue

Paige, NYC, NY United States(美国纽约市帕伊公司);Microsoft Research, Cambridge, MA United States(微软研究院);Memorial Sloan Kettering Cancer Center, NYC, NY United States(纪念斯隆凯特林癌症中心);University of Yale, New Haven, CT United States(耶鲁大学)
Eugene Vorontsov, George Shaikovski, Adam Casson, Julian Viret, Eric Zimmermann, Neil Tenenholtz, Yi Kan Wang, Jan H. Bernhard, Ran A. Godrich, Juan A. Retamero… 展开作者

Eugene Vorontsov, George Shaikovski, Adam Casson, Julian Viret, Eric Zimmermann, Neil Tenenholtz, Yi Kan Wang, Jan H. Bernhard, Ran A. Godrich, Juan A. Retamero, Jinru Shia, Mithat Gonen, Martin R. Weiser, David S. Klimstra, Razik Yousfi, Nicolo Fusi, Thomas J. Fuchs, Kristen Severson, Siqi Liu

AI总结:

PRISM2是一个基于临床对话监督训练的多模态切片级基础模型,利用最大组织病理学数据集对齐形态特征与诊断推理,在癌症检测等任务上无需额外训练即超越临床级产品,并通过微调进一步提升性能。

摘要:

基础模型推动了计算病理学领域的近期快速发展。这些模型正开始从编码图像块向全切片理解发展,但其临床实用性仍然有限。在本研究中,我们提出了PRISM2,这是一个多模态切片级基础模型,在700,000个诊断标本-报告对的数据上进行了训练,这是迄今为止最大的视觉(230万张全切片图像)和语言(1400万个问答对)组织病理学数据集。通过临床对话监督进行学习,PRISM2将组织形态学特征与诊断推理的语言对齐,生成支持直接诊断问答和下游任务可迁移嵌入的切片级表示。无需额外训练,PRISM2就能匹配或超越临床级产品的癌症检测性能。在其他任务上同样如此,PRISM2取得了顶尖表现。最后,以生存预测为例,我们展示了使用大型数据集进行任务特定微调可以超越任务特定模型,进一步提升性能。这些结果表明,语言监督预训练为学习可泛化的病理表示提供了可扩展且具有临床基础的信号,架起了人类诊断推理与基础模型性能之间的桥梁。

英文摘要:

Recent rapid progress in the field of computational pathology has been enabled by foundation models. These models are beginning to move beyond encoding image patches towards whole-slide understanding but their clinical utility remains limited. In this work, we present PRISM2, a multimodal slide-level foundation model trained on data from 700,000 diagnostic specimen-report pairs, the largest vision (2.3 million whole slide images) and language (14M question-answer pairs) histopathology dataset to date. By learning through clinical-dialogue supervision, PRISM2 aligns histomorphologic features with the language of diagnostic reasoning, producing slide-level representations that support both direct diagnostic question-answering and transferable embeddings for downstream tasks. Without additional training, PRISM2 matches or exceeds the cancer-detection performance of clinical-grade products. This is observed without loss of generality on other tasks, where PRISM2 achieves top performance. Finally, using survival prediction as the example, we show that task-specific finetuning with a large dataset can outperform task-specific models, further improving performance. These results demonstrate how language-supervised pretraining provides a scalable, clinically grounded signal for learning generalizable pathology representations, bridging human diagnostic reasoning and foundation-model performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑