MaskSAM：面向体医学图像分割的基于掩码分类的自动提示SAM
MaskSAM: Towards Auto-prompt SAM with Mask Classification for Volumetric Medical Image Segmentation
AI总结：
针对SAM直接用于医学图像分割存在的语义标签预测缺失、需额外提示、性能不佳问题，提出无提示的MaskSAM框架，通过提示生成器、分类器token及3D适配器微调SAM，在多个数据集上超越nnUNet，实现SOTA性能。
摘要：
Segment Anything Model（SAM，分割一切模型）是一种用于自然图像分割的提示驱动基础模型，已展现出令人瞩目的零样本性能。然而，SAM直接应用于医学图像分割时无法有效工作，原因在于SAM缺乏预测语义标签的能力、需要额外提示，且性能表现不佳。针对上述问题，我们提出MaskSAM，这是一种用于医学图像分割的新型无提示掩码分类SAM适配框架。我们设计了一个与SAM中的图像编码器结合的提示生成器，用于生成一组辅助分类器token、辅助二值掩码和辅助边界框。每对辅助掩码和框提示可解决额外提示的需求。语义标签预测可通过SAM掩码解码器中的辅助分类器token与可学习全局分类器token相加来解决。同时，我们设计了用于图像嵌入的3D深度卷积适配器和用于提示嵌入的3D深度MLP适配器，以高效微调SAM。我们的方法在AMOS2022数据集上达到了90.52%的Dice指标，实现了当前最优性能，相比nnUNet提升了2.7%。在ACDC数据集上，我们的方法比nnUNet高出1.7%，在Synapse数据集上高出1.0%。
英文摘要：
Segment Anything Model (SAM), a prompt-driven foundation model for natural image segmentation, has demonstrated impressive zero-shot performance. However, SAM does not work when directly applied to medical image segmentation, since SAM lacks the ability to predict semantic labels, requires additional prompts, and presents suboptimal performance. Following the above issues, we propose MaskSAM, a novel mask classification prompt-free SAM adaptation framework for medical image segmentation. We design a prompt generator combined with the image encoder in SAM to generate a set of auxiliary classifier tokens, auxiliary binary masks, and auxiliary bounding boxes. Each pair of auxiliary mask and box prompts can solve the requirements of extra prompts. The semantic label prediction can be addressed by the sum of the auxiliary classifier tokens and the learnable global classifier tokens in the mask decoder of SAM. Meanwhile, we design a 3D depth-convolution adapter for image embeddings and a 3D depth-MLP adapter for prompt embeddings to efficiently fine-tune SAM. Our method achieves state-of-the-art performance on AMOS2022, 90.52% Dice, which improved by 2.7% compared to nnUNet. Our method surpasses nnUNet by 1.7% on ACDC and 1.0% on Synapse datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
