arXiv 2403.158362025-07-29cs.CV

VLM-CPL：来自视觉-语言模型的一致性伪标签用于无标注病理图像分类

VLM-CPL: Consensus Pseudo Labels from Vision-Language Models for Annotation-Free Pathological Image Classification

School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China（电子科技大学机械与电子工程学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Department of Pathology, Sichuan Clinical Research Center for Cancer, Sichuan Cancer Hospital & Institute, Affiliated Cancer Hospital of University of Electronic Science and Technology of China（pathology department, 四川省癌症临床研究中心, 四川省肿瘤医院及研究所, 电子科技大学附属肿瘤医院）；Department of Radiation Oncology, Sichuan Cancer Hospital and Institute, University of Electronic Science and Technology of China（放射肿瘤科, 四川省肿瘤医院及研究所, 电子科技大学）

Lanfeng Zhong, Zongyao Huang, Yang Liu, Wenjun Liao, Shichuan Zhang, Guotai Wang, Shaoting Zhang

备注：: Accepted at TMI

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对病理图像分类依赖人工标注的问题，提出VLM-CPL方法，通过VLM生成两类伪标签并基于一致性筛选可靠样本，结合半监督学习与开集提示策略，实现无人工标注下性能优于现有方法的病理图像分类。

摘要：

病理图像分类是癌症自动诊断的基础。尽管深度学习方法已取得显著性能，但它们严重依赖标注数据，需要大量的人工标注工作。在本研究中，我们利用预训练视觉-语言模型（Vision-Language Models, VLMs）提出了一种无需人工标注的新方法。在无人工标注的情况下，利用VLM的零样本推理能力可获得训练集的伪标签，但由于预训练数据集与目标数据集之间存在域差距，这些伪标签可能包含大量噪声。为解决这一问题，我们提出了VLM-CPL这一新方法，该方法包含两种噪声标签过滤技术，并结合了半监督学习策略。具体而言，我们首先利用VLM对输入的多个增强视图进行零样本推理，得到带不确定性估计的基于提示的伪标签。随后，利用VLM的特征表示能力，通过特征空间中的样本聚类得到基于特征的伪标签。我们引入提示-特征一致性机制，基于两类伪标签之间的一致性来筛选可靠样本。我们进一步提出高置信度交叉监督方法，以从带有可靠伪标签的样本和剩余未标注样本中学习。此外，我们提出了一种创新的开集提示策略，可从全切片中过滤不相关的图像块，以提升所选图像块的质量。在五个用于图像块级别和切片级别分类的公开病理图像数据集上的实验结果表明，我们的方法大幅优于VLM的零样本分类，且优于现有的噪声标签学习方法。代码已公开在https://github.com/HiLab-git/VLM-CPL。

英文摘要：

Classification of pathological images is the basis for automatic cancer diagnosis. Despite that deep learning methods have achieved remarkable performance, they heavily rely on labeled data, demanding extensive human annotation efforts. In this study, we present a novel human annotation-free method by leveraging pre-trained Vision-Language Models (VLMs). Without human annotation, pseudo-labels of the training set are obtained by utilizing the zero-shot inference capabilities of VLM, which may contain a lot of noise due to the domain gap between the pre-training and target datasets. To address this issue, we introduce VLM-CPL, a novel approach that contains two noisy label filtering techniques with a semi-supervised learning strategy. Specifically, we first obtain prompt-based pseudo-labels with uncertainty estimation by zero-shot inference with the VLM using multiple augmented views of an input. Then, by leveraging the feature representation ability of VLM, we obtain feature-based pseudo-labels via sample clustering in the feature space. Prompt-feature consensus is introduced to select reliable samples based on the consensus between the two types of pseudo-labels. We further propose High-confidence Cross Supervision by to learn from samples with reliable pseudo-labels and the remaining unlabeled samples. Additionally, we present an innovative open-set prompting strategy that filters irrelevant patches from whole slides to enhance the quality of selected patches. Experimental results on five public pathological image datasets for patch-level and slide-level classification showed that our method substantially outperformed zero-shot classification by VLMs, and was superior to existing noisy label learning methods. The code is publicly available at https://github.com/HiLab-git/VLM-CPL.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。