arXiv 2403.125372026-07-21cs.CV

用于病理图像分类的提示引导基础模型调优

Prompt-Guided Foundation Model Tuning for Pathology Image Classification

Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（香港科技大学电子与计算机工程系）；Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology（香港科技大学化学与生物工程系）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深圳-香港协同创新研究院）；State Key Laboratory of Nervous System Disorders, The Hong Kong University of Science and Technology（香港科技大学神经系统疾病国家重点实验室）

Yi Lin, Zhengjie Zhu, Kwang-Ting Cheng, Hao Chen

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

研究针对病理图像分类中基础模型预训练与下游任务的差异问题，提出PAMT框架，通过引入RPS、PVP及AMT，在14个公开数据集上严格评估，显著提升分类准确率，确立了PAMT作为病理图像分类新基准的地位。

摘要：

基础模型在推进计算病理学方面至关重要，特别是对于全切片图像（WSI）分类。然而，现有方法常依赖冻结的预训练模型进行特征提取，忽视了预训练与下游任务间显著的领域转移和任务差异。为此提出PAMT，一种新颖的提示引导自适应模型转换框架，能使通用基础模型精确适应组织病理学的不同领域。引入代表性补丁采样（RPS）和原型视觉提示（PVP）来封装组织病理学数据的复杂分布特征，通过特征提取管道中的适配器模块纳入自适应模型转换（AMT）以有效弥合领域差距。在14个公开数据集上进行严格评估，分类准确率有持续显著提高。这些结果确立了PAMT作为病理图像分类的新基准，并强调了计算病理学中目标模型适应的关键价值。

英文摘要：

Foundation models have become pivotal in advancing computational pathology, particularly for whole slide image (WSI) classification. However, prevailing methodologies often rely on frozen, pre-trained models for feature extraction, overlooking the pronounced domain shift and task discrepancy between the pre-training and downstream tasks. To address this challenge, we propose PAMT, a novel Prompt-guided Adaptive Model Transformation framework that enables precise adaptation of general foundation models to the distinct domain of histopathology. To encapsulate the intricate distributions characteristic of histopathological data, we introduce Representative Patch Sampling (RPS) and Prototypical Visual Prompt (PVP), which reconstruct the input into compact yet highly informative representations. Further, to effectively bridge the domain gap, we incorporate Adaptive Model Transformation (AMT) via adapter modules within the feature extraction pipeline, facilitating the acquisition of domain-specific features by the foundation model. We conduct rigorous evaluation across 14 publicly available datasets and demonstrate consistent, substantial improvements in classification accuracy. These results establish PAMT as a compelling new benchmark for pathology image classification and underscore the critical value of targeted model adaptation within computational pathology.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。