通过弱少样本监督学习提示实现MedSAM自动化
Automating MedSAM by Learning Prompts with Weak Few-Shot Supervision
AI总结：
本文提出用轻量级模块从图像嵌入中直接学习提示嵌入，替代SAM类基础模型对手工提示的依赖，并结合弱标签和10样本少样本监督，在MedSAM上实现医学图像自动分割。
摘要：
基础模型（如最近提出的Segment Anything Model (SAM)）在图像分割任务中取得了显著成果。然而，这些模型通常需要通过手工设计的提示（如边界框）进行用户交互，这限制了它们在下游任务中的部署。使用完全标注的数据将这些模型适配到特定任务还需要昂贵的先验用户交互来获取真实标注。本文提出用一个轻量级模块替代对输入提示的条件依赖，该模块直接从图像嵌入中学习提示嵌入，随后基础模型使用这两者输出分割掩码。我们具有可学习提示的基础模型能够自动分割任意特定区域，方法是：1）通过一个简单模块预测的提示嵌入来修改输入，以及2）使用弱标签（紧边界框）和少样本监督（10个样本）。我们的方法在MedSAM（一个针对医学图像微调的SAM版本）上进行了验证，并在MR和超声成像的三个医学数据集上取得了结果。我们的代码可在https://github.com/Minimel/MedSAMWeakFewShotPromptAutomation获取。
英文摘要：
Foundation models such as the recently introduced Segment Anything Model (SAM) have achieved remarkable results in image segmentation tasks. However, these models typically require user interaction through handcrafted prompts such as bounding boxes, which limits their deployment to downstream tasks. Adapting these models to a specific task with fully labeled data also demands expensive prior user interaction to obtain ground-truth annotations. This work proposes to replace conditioning on input prompts with a lightweight module that directly learns a prompt embedding from the image embedding, both of which are subsequently used by the foundation model to output a segmentation mask. Our foundation models with learnable prompts can automatically segment any specific region by 1) modifying the input through a prompt embedding predicted by a simple module, and 2) using weak labels (tight bounding boxes) and few-shot supervision (10 samples). Our approach is validated on MedSAM, a version of SAM fine-tuned for medical images, with results on three medical datasets in MR and ultrasound imaging. Our code is available on https://github.com/Minimel/MedSAMWeakFewShotPromptAutomation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
