AI语言病理学家的崛起：探索用于少样本弱监督全切片图像分类的两级提示学习
The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classification
AI总结：
针对病理学全切片图像少样本弱监督分类难题，提出融合GPT-4语言先验的两级提示学习多示例学习框架，在三类癌症数据集上实现了包与示例级分类的性能提升。
摘要：
本文提出了病理学全切片图像（Whole Slide Image，WSI）分类的少样本弱监督学习新概念，记为FSWC，并基于提示学习与大语言模型GPT-4的应用提出了一种解决方案。由于WSI尺寸过大，需要分割为图像块（patch）进行处理，因此WSI分类通常被建模为多示例学习（Multiple Instance Learning，MIL）问题：在该框架下，每张WSI被视为一个包（bag），分割得到的图像块被视为示例（instance）。FSWC的目标是仅利用少量带标签的包，同时实现包级别和示例级别的分类。与传统少样本学习问题不同，MIL框架下的包标签为弱标签，这给FSWC带来了额外挑战。受视觉-语言模型（V-L模型）在下游少样本分类任务中最新进展的启发，我们提出了一种适配病理学场景的两级提示学习MIL框架，该框架融合了语言先验知识。具体而言，我们利用CLIP提取每个图像块的示例特征，并提出一种提示引导的池化策略，将这些示例特征聚合为包特征。随后，我们基于包特征，使用少量带标签的包开展少样本提示学习。我们的方法通过问答模式调用GPT-4，获取示例级别和包级别的语言先验知识，并将这些知识整合到对应层级的语言提示中；此外，语言提示的可学习组件通过可用的少样本标注数据进行训练。我们在包含乳腺癌、肺癌、宫颈癌的三个真实WSI数据集上开展了大量实验，结果表明所提方法在包分类和示例分类任务上均取得了显著性能。所有代码将公开提供。
英文摘要：
This paper introduces the novel concept of few-shot weakly supervised learning for pathology Whole Slide Image (WSI) classification, denoted as FSWC. A solution is proposed based on prompt learning and the utilization of a large language model, GPT-4. Since a WSI is too large and needs to be divided into patches for processing, WSI classification is commonly approached as a Multiple Instance Learning (MIL) problem. In this context, each WSI is considered a bag, and the obtained patches are treated as instances. The objective of FSWC is to classify both bags and instances with only a limited number of labeled bags. Unlike conventional few-shot learning problems, FSWC poses additional challenges due to its weak bag labels within the MIL framework. Drawing inspiration from the recent achievements of vision-language models (V-L models) in downstream few-shot classification tasks, we propose a two-level prompt learning MIL framework tailored for pathology, incorporating language prior knowledge. Specifically, we leverage CLIP to extract instance features for each patch, and introduce a prompt-guided pooling strategy to aggregate these instance features into a bag feature. Subsequently, we employ a small number of labeled bags to facilitate few-shot prompt learning based on the bag features. Our approach incorporates the utilization of GPT-4 in a question-and-answer mode to obtain language prior knowledge at both the instance and bag levels, which are then integrated into the instance and bag level language prompts. Additionally, a learnable component of the language prompts is trained using the available few-shot labeled data. We conduct extensive experiments on three real WSI datasets encompassing breast cancer, lung cancer, and cervical cancer, demonstrating the notable performance of the proposed method in bag and instance classification. All codes will be available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
