arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2601.10880cs.CVcs.AI

医学SAM3:一种通用提示驱动的医学图像分割基础模型

Medical SAM3: A Foundation Model for Universal Prompt-Driven Medical Image Segmentation

University of Central Florida, Orlando, USA(佛罗里达大学);University College London, London, UK(伦敦大学学院);University of Illinois Urbana-Champaign, Champaign, USA(伊利诺伊大学厄巴纳-香槟分校);University of Trento, Trento, Italy(特伦托大学);The Hong Kong Polytechnic University, China(香港理工大学);University of Pennsylvania, Philadelphia, USA(宾夕法尼亚大学)

Chongcong Jiang, Tianxingjian Ding, Chuhan Song, Jiachen Tu, Ziyang Yan, Yihua Shao, Zhenyi Wang, Yuzhang Shang, Tianyu Han, Yu Tian

AI总结:

Medical SAM3通过微调SAM3模型,提升医学图像分割的通用提示驱动能力,解决领域偏移和复杂结构推理问题。

摘要:

可提示的分割基础模型,如SAM3,通过交互式和基于概念的提示展示了强大的泛化能力。然而,它们直接应用于医学图像分割仍然受到严重的领域偏移、缺乏特权空间提示以及需要推理复杂解剖和体积结构的限制。本文提出Medical SAM3,一种通用提示驱动的医学图像分割基础模型,通过在大规模、异质的2D和3D医学影像数据集上完全微调SAM3,这些数据集配有配对的分割掩码和文本提示。通过系统分析原始SAM3,我们发现其在医学数据上的性能显著下降,其明显竞争力主要依赖于强大的几何先验,如基于真实情况的边界框。这些发现促使我们超越仅提示工程进行完整的模型适应。通过在涵盖10种医学影像模态的33个数据集上微调SAM3的模型参数,Medical SAM3获得了稳健的领域特定表示,同时保持了提示驱动的灵活性。在器官、影像模态和维度上进行的广泛实验表明,一致且显著的性能提升,特别是在具有语义模糊、复杂形态和长距离3D上下文的挑战性场景中。我们的结果确立了Medical SAM3作为医学影像的通用、文本引导的分割基础模型,并突显了全面模型适应在实现严重领域偏移下的稳健提示驱动分割中的重要性。代码和模型将在https://github.com/AIM-Research-Lab/Medical-SAM3上提供。

英文摘要:

Promptable segmentation foundation models such as SAM3 have demonstrated strong generalization capabilities through interactive and concept-based prompting. However, their direct applicability to medical image segmentation remains limited by severe domain shifts, the absence of privileged spatial prompts, and the need to reason over complex anatomical and volumetric structures. Here we present Medical SAM3, a foundation model for universal prompt-driven medical image segmentation, obtained by fully fine-tuning SAM3 on large-scale, heterogeneous 2D and 3D medical imaging datasets with paired segmentation masks and text prompts. Through a systematic analysis of vanilla SAM3, we observe that its performance degrades substantially on medical data, with its apparent competitiveness largely relying on strong geometric priors such as ground-truth-derived bounding boxes. These findings motivate full model adaptation beyond prompt engineering alone. By fine-tuning SAM3's model parameters on 33 datasets spanning 10 medical imaging modalities, Medical SAM3 acquires robust domain-specific representations while preserving prompt-driven flexibility. Extensive experiments across organs, imaging modalities, and dimensionalities demonstrate consistent and significant performance gains, particularly in challenging scenarios characterized by semantic ambiguity, complex morphology, and long-range 3D context. Our results establish Medical SAM3 as a universal, text-guided segmentation foundation model for medical imaging and highlight the importance of holistic model adaptation for achieving robust prompt-driven segmentation under severe domain shift. Code and model will be made available at https://github.com/AIM-Research-Lab/Medical-SAM3.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑