arXiv 2501.162462025-01-27cs.CV

CLISC：通过增强的CAM连接CLIP和SAM以进行无监督脑肿瘤分割

CLISC: Bridging clip and sam by enhanced cam for unsupervised brain tumor segmentation

School of Computer Science and Engineering（计算机科学与工程学院）；School of Mechanical and Electrical Engineering（机械与电气工程学院）；University of Electronic Science and Technology of China（电子科技大学）；School of Medicine, University of Electronic Science and Technology of China（电子科技大学医学院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）

Xiaochuan Ma, Jia Fu, Wenjun Liao, Shichuan Zhang, Guotai Wang

备注：: 22st IEEE International Symposium on Biomedical Imaging (ISBI 2025)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出一种利用CLIP、CAM和SAM的无监督脑肿瘤分割方法，通过生成并过滤伪标签训练3D网络，在BraTS2020上DSC达85.60%，性能接近全监督学习。

摘要：

脑肿瘤分割对肿瘤诊断很重要，当前深度学习方法依赖大量标注图像进行训练，标注成本高。无监督分割有望避免人工标注，但性能常受限。本研究提出一种利用基础模型能力的新型无监督分割方法，包含三个主要步骤：(1) 采用视觉-语言模型（即CLIP）获取图像级伪标签以训练分类网络。随后采用类激活映射（CAM）提取感兴趣区域（ROIs），其中使用基于自适应掩码的数据增强来增强ROI识别。(2) 利用ROIs生成边界框和点提示，供Segment Anything Model（SAM）使用以获取分割伪标签。(3) 使用SAM生成的伪标签训练3D分割网络，在此过程中，基于SAM输出与网络预测间的相似性，通过自学习过程过滤低质量伪标签。在BraTS2020数据集上的评估表明，该方法获得了85.60%的平均Dice相似性得分（DSC），以超过10个百分点的优势优于五种最先进的无监督分割方法。此外，该方法优于直接使用SAM进行零样本推理，且其性能接近全监督学习。

英文摘要：

Brain tumor segmentation is important for diagnosis of the tumor, and current deep-learning methods rely on a large set of annotated images for training, with high annotation costs. Unsupervised segmentation is promising to avoid human annotations while the performance is often limited. In this study, we present a novel unsupervised segmentation approach that leverages the capabilities of foundation models, and it consists of three main steps: (1) A vision-language model (i.e., CLIP) is employed to obtain image-level pseudo-labels for training a classification network. Class Activation Mapping (CAM) is then employed to extract Regions of Interest (ROIs), where an adaptive masking-based data augmentation is used to enhance ROI identification.(2) The ROIs are used to generate bounding box and point prompts for the Segment Anything Model (SAM) to obtain segmentation pseudo-labels. (3) A 3D segmentation network is trained with the SAM-derived pseudo-labels, where low-quality pseudo-labels are filtered out in a self-learning process based on the similarity between the SAM's output and the network's prediction. Evaluation on the BraTS2020 dataset demonstrates that our approach obtained an average Dice Similarity Score (DSC) of 85.60%, outperforming five state-of-the-art unsupervised segmentation methods by more than 10 percentage points. Besides, our approach outperforms directly using SAM for zero-shot inference, and its performance is close to fully supervised learning.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。