arXiv 2307.129142023-07-25cs.CVcs.AI

迈向计算病理学的视觉-语言基础模型

Towards a Visual-Language Foundation Model for Computational Pathology

Brigham and Women’s Hospital（布莱根妇女医院）；Harvard Medical School（哈佛医学院）；Massachusetts General Hospital（麻省总医院）；Broad Institute of Harvard and MIT（哈佛-麻省理工学院博德研究所）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Massachusetts Institute of Technology (MIT)（麻省理工学院）；Harvard-MIT（哈佛-麻省理工学院）；Harvard John A. Paulson School of Engineering And Applied Sciences, Harvard University（哈佛大学约翰·A·保尔森工程与应用科学学院）；Wexner Medical Center, Ohio State University（俄亥俄州立大学韦克斯纳医学中心）；Harvard Data Science Initiative, Harvard University（哈佛大学数据科学计划）

Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Andrew Zhang, Long Phi Le, Georg Gerber, A… 展开作者

Ming Y. Lu, Bowen Chen, Drew F. K. Williamson, Richard J. Chen, Ivy Liang, Tong Ding, Guillaume Jaume, Igor Odintsov, Andrew Zhang, Long Phi Le, Georg Gerber, Anil V Parwani, Faisal Mahmood

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出CONCH视觉-语言基础模型，利用117万图像-文本对预训练，在13项基准上实现组织病理学分类、分割、检索等任务的最优性能，减少监督微调需求。

摘要：

数字病理学的加速应用和深度学习的进步，使得开发针对多种疾病和患者队列的各种病理学任务的强大模型成为可能。然而，由于医学领域标签稀缺，模型训练往往困难，且模型的使用受限于其训练时针对的特定任务和疾病。此外，组织病理学中的大多数模型仅利用图像数据，这与人类相互教学和推理组织病理学实体的方式形成鲜明对比。我们引入了用于组织病理学的基于标题的对比学习（CONCH），这是一个利用多样来源的组织病理学图像、生物医学文本以及通过任务无关预训练获得的超过117万图像-标题对开发的视觉-语言基础模型。在13个多样化基准测试套件上评估，CONCH可迁移到涉及组织病理学图像和/或文本的广泛下游任务，在组织学图像分类、分割、标题生成、文本到图像和图像到文本检索方面达到了最先进的性能。CONCH代表了与同期视觉-语言预训练系统相比在组织病理学领域的重大飞跃，有潜力直接促进需要极少或无需进一步监督微调的各种基于机器学习的工作流程。

英文摘要：

The accelerated adoption of digital pathology and advances in deep learning have enabled the development of powerful models for various pathology tasks across a diverse array of diseases and patient cohorts. However, model training is often difficult due to label scarcity in the medical domain and the model's usage is limited by the specific task and disease for which it is trained. Additionally, most models in histopathology leverage only image data, a stark contrast to how humans teach each other and reason about histopathologic entities. We introduce CONtrastive learning from Captions for Histopathology (CONCH), a visual-language foundation model developed using diverse sources of histopathology images, biomedical text, and notably over 1.17 million image-caption pairs via task-agnostic pretraining. Evaluated on a suite of 13 diverse benchmarks, CONCH can be transferred to a wide range of downstream tasks involving either or both histopathology images and text, achieving state-of-the-art performance on histology image classification, segmentation, captioning, text-to-image and image-to-text retrieval. CONCH represents a substantial leap over concurrent visual-language pretrained systems for histopathology, with the potential to directly facilitate a wide array of machine learning-based workflows requiring minimal or no further supervised fine-tuning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。