arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.15028cs.CV
收藏

PRISM:一种基于视觉提示的可提示且鲁棒的交互式分割模型

PRISM: A Promptable and Robust Interactive Segmentation Model with Visual Prompts

Vanderbilt University(范德堡大学)

Hao Li, Han Liu, Dewei Hu, Jiacheng Wang, Ipek Oguz

AI总结:

PRISM是一种基于视觉提示的3D医学图像交互式分割模型,通过迭代、置信度、纠正和混合设计四项原则提升鲁棒性,在多器官肿瘤分割任务中显著超越现有方法并接近人类水平。

摘要:

本文提出了PRISM,一种可提示且鲁棒的交互式分割模型,旨在对3D医学图像进行精确分割。PRISM接受多种视觉输入,包括将点、框和涂鸦作为稀疏提示,以及将掩码作为密集提示。具体而言,PRISM基于四项原则设计以实现鲁棒性:(1)迭代学习。模型利用先前迭代的视觉提示生成分割,以实现逐步改进。(2)置信度学习。PRISM为每张输入图像采用多个分割头,每个分割头生成连续图和置信度分数以优化预测。(3)纠正学习。在每次分割迭代后,PRISM采用浅层纠正细化网络重新分配错误标记的体素。(4)混合设计。PRISM集成混合编码器以更好地捕获局部和全局信息。使用四个公开数据集对结肠、胰腺、肝脏和肾脏中的肿瘤分割进行了全面验证,突出了解剖变异和模糊边界在准确肿瘤识别中造成的挑战。与最先进的方法相比,无论是否进行提示工程,PRISM都显著提高了性能,取得了接近人类水平的结果。代码公开于https://github.com/MedICL-VU/PRISM。

英文摘要:

In this paper, we present PRISM, a Promptable and Robust Interactive Segmentation Model, aiming for precise segmentation of 3D medical images. PRISM accepts various visual inputs, including points, boxes, and scribbles as sparse prompts, as well as masks as dense prompts. Specifically, PRISM is designed with four principles to achieve robustness: (1) Iterative learning. The model produces segmentations by using visual prompts from previous iterations to achieve progressive improvement. (2) Confidence learning. PRISM employs multiple segmentation heads per input image, each generating a continuous map and a confidence score to optimize predictions. (3) Corrective learning. Following each segmentation iteration, PRISM employs a shallow corrective refinement network to reassign mislabeled voxels. (4) Hybrid design. PRISM integrates hybrid encoders to better capture both the local and global information. Comprehensive validation of PRISM is conducted using four public datasets for tumor segmentation in the colon, pancreas, liver, and kidney, highlighting challenges caused by anatomical variations and ambiguous boundaries in accurate tumor identification. Compared to state-of-the-art methods, both with and without prompt engineering, PRISM significantly improves performance, achieving results that are close to human levels. The code is publicly available at https://github.com/MedICL-VU/PRISM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑