arXiv 2502.148072025-10-19eess.IVcs.AIcs.CV

FetalCLIP：用于胎儿超声图像分析的视觉-语言基础模型

FetalCLIP: A Visual-Language Foundation Model for Fetal Ultrasound Image Analysis

Department of Computer Vision（计算机视觉系）；Mohamed bin Zayed University of Artificial Intelligence（马尔代夫比兹人工智能大学）；Department of Machine Learning（机器学习系）；Corniche Hospital, Abu Dhabi Health Services Company (SEHA)（阿布扎赫尔医院，阿布扎赫健康服务公司（SEHA））

Fadillah Maani, Numan Saeed, Tausifa Saleem, Zaid Farooq, Hussain Alasmawi, Werner Diehl, Ameera Mohammad, Gareth Waring, Saudabi Valappi, Leanne Bricker, Mohammad Yaqub

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出视觉-语言基础模型FetalCLIP，基于210,035张图文配对胎儿超声图像进行多模态预训练，在分类、胎龄估计、CHD检测和分割任务上超越基线。

摘要：

基础模型在医学领域正变得越来越有效，其提供在大规模数据集上预训练的模型，可方便地适配下游任务。尽管取得了进展，由于胎儿超声图像固有的复杂性，其对基础模型而言仍是一个具有挑战性的领域，通常需要大量额外训练，并因成对多模态数据稀缺而面临限制。为克服这些挑战，我们在此提出FetalCLIP，这是一种能够生成胎儿超声图像通用表征的视觉-语言基础模型。FetalCLIP采用多模态学习方法，在包含210,035张与文本配对的胎儿超声图像的多样化数据集上进行预训练。这是迄今为止用于基础模型开发的同类最大规模成对数据集。这种独特的训练方法使FetalCLIP能够有效学习胎儿超声图像中复杂的解剖特征，从而产生可用于多种下游应用的鲁棒表征。在一系列关键胎儿超声应用（包括分类、胎龄估计、先天性心脏缺陷（CHD）检测和胎儿结构分割）的广泛基准测试中，FetalCLIP优于所有基线，同时展现出显著的泛化能力，即使在标注数据有限的情况下也具有强劲性能。我们计划公开发布FetalCLIP模型，以惠及更广泛的科学界。

英文摘要：

Foundation models are becoming increasingly effective in the medical domain, offering pre-trained models on large datasets that can be readily adapted for downstream tasks. Despite progress, fetal ultrasound images remain a challenging domain for foundation models due to their inherent complexity, often requiring substantial additional training and facing limitations due to the scarcity of paired multimodal data. To overcome these challenges, here we introduce FetalCLIP, a vision-language foundation model capable of generating universal representation of fetal ultrasound images. FetalCLIP was pre-trained using a multimodal learning approach on a diverse dataset of 210,035 fetal ultrasound images paired with text. This represents the largest paired dataset of its kind used for foundation model development to date. This unique training approach allows FetalCLIP to effectively learn the intricate anatomical features present in fetal ultrasound images, resulting in robust representations that can be used for a variety of downstream applications. In extensive benchmarking across a range of key fetal ultrasound applications, including classification, gestational age estimation, congenital heart defect (CHD) detection, and fetal structure segmentation, FetalCLIP outperformed all baselines while demonstrating remarkable generalizability and strong performance even with limited labeled data. We plan to release the FetalCLIP model publicly for the benefit of the broader scientific community.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。