arXiv 2501.037222025-01-07cs.CVcs.AI

用于肺动脉和静脉3D分割的自适应视觉-语言模型

Self-adaptive vision-language model for 3D segmentation of pulmonary artery and vein

National Clinical Research Center for Cancer/Cancer Hospital Shenzhen Hospital（国家癌症临床研究中心/肿瘤医院深圳医院）；Key Laboratory of System Software (Chinese Academy of Sciences)（系统软件重点实验室（中国科学院））；State Key Laboratory of Computer Science（计算机科学国家重点实验室）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；Guangzhou Jiayi Software Technology Co., Ltd.（广州嘉益软件科技有限公司）；R&D Center, Guangxi Huayi Artificial Intelligence Medical Technology Co., Ltd（广西华奕人工智能医疗科技有限公司研发中心）；Department of Computer Science, University of Exeter（埃克塞特大学计算机科学系）；Department of Cardiovascular & Metabolic Medicine, University of Liverpool（利物浦大学心血管与代谢医学系）

Xiaotong Guo, Deqian Yang, Dan Wang, Haochen Zhao, Yuan Li, Zhilin Sui, Tao Zhou, Lijun Zhang, Yanda Meng

备注：: 8 pages,3 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对肺动静脉3D分割依赖大量标注数据的问题，提出基于预训练CLIP的语言引导自适应交叉注意力融合框架，在718例最大CT数据集上显著优于现有方法。

摘要：

肺部结构的准确分割在临床诊断、疾病研究和治疗规划中至关重要。基于深度学习的分割技术已取得显著进展，但大多数技术需要大量标注数据进行训练。因此，在医学图像分析中，开发需要较少标注数据集的精确分割方法至关重要。诸如CLIP之类的预训练视觉-语言基础模型的出现，最近为通用计算机视觉任务打开了大门。在分割等下游任务上利用这些预训练基础模型的泛化能力，能够以相对少量的标注数据获得出乎意料的性能。然而，探索这些模型用于肺动静脉分割仍然有限。本文提出了一种名为语言引导的自适应交叉注意力融合框架的新颖框架。我们的方法采用预训练CLIP作为强大的特征提取器，用于生成3D CT扫描的分割，同时自适应地聚合文本和图像表示的跨模态信息。我们提出了一个专门设计的适配器模块，通过自适应学习策略微调预训练CLIP，以有效融合两种模态的嵌入。我们在一个本地数据集上广泛验证了我们的方法，该数据集是迄今为止最大的肺动静脉CT数据集，总共包含718个标注数据。实验表明，我们的方法大幅优于其他最先进的方法。我们的数据和代码将在论文被接收后公开发布。

英文摘要：

Accurate segmentation of pulmonary structures iscrucial in clinical diagnosis, disease study, and treatment planning. Significant progress has been made in deep learning-based segmentation techniques, but most require much labeled data for training. Consequently, developing precise segmentation methods that demand fewer labeled datasets is paramount in medical image analysis. The emergence of pre-trained vision-language foundation models, such as CLIP, recently opened the door for universal computer vision tasks. Exploiting the generalization ability of these pre-trained foundation models on downstream tasks, such as segmentation, leads to unexpected performance with a relatively small amount of labeled data. However, exploring these models for pulmonary artery-vein segmentation is still limited. This paper proposes a novel framework called Language-guided self-adaptive Cross-Attention Fusion Framework. Our method adopts pre-trained CLIP as a strong feature extractor for generating the segmentation of 3D CT scans, while adaptively aggregating the cross-modality of text and image representations. We propose a s pecially designed adapter module to fine-tune pre-trained CLIP with a self-adaptive learning strategy to effectively fuse the two modalities of embeddings. We extensively validate our method on a local dataset, which is the largest pulmonary artery-vein CT dataset to date and consists of 718 labeled data in total. The experiments show that our method outperformed other state-of-the-art methods by a large margin. Our data and code will be made publicly available upon acceptance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。