arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.13908cs.CV

记忆化SAM:基于记忆化Transformer的3D医学图像分割任意模型

Memorizing SAM: 3D Medical Segment Anything Model with Memorizing Transformer

Johns Hopkins University(约翰斯·霍普金斯大学)

Xinyuan Shao, Yiqing Shen, Mathias Unberath

AI总结:

本文提出Memorizing SAM,通过将记忆Transformer作为插件引入3D SAM架构,利用高精度内部表示作为记忆源,在有限计算成本下显著提升了3D医学图像分割性能,平均Dice增加11.36%且推理仅增4.38毫秒。

摘要:

分割任意模型(SAM)因其在给定适当用户提示时对未见类别和领域的对象具备零样本泛化能力,在医学图像分析中受到越来越多的关注。弥补这一性能差距对于充分利用SAM的预训练权重至关重要,特别是在体积医学图像分割领域,该领域准确性很重要,但用于微调的标注良好的3D医学数据有限。在本研究中,我们探讨了将记忆机制作为插件引入,即记忆和回忆过去输入内部表示的能力,能否以有限的计算成本提升SAM的性能。为此,我们提出了Memorizing SAM,这是一种结合记忆Transformer作为插件的新型3D SAM架构。与在训练或推理期间保存内部表示的传统记忆化Transformer不同,我们的Memorizing SAM利用现有的高精度内部表示作为记忆源,以确保记忆质量。我们在TotalSegmentator数据集的33个类别中评估了Memorizing SAM的性能,结果表明Memorizing SAM优于最先进的3D SAM变体,即FastSAM3D,平均Dice增加11.36%,而推理时间仅增加4.38毫秒。源代码已在https://github.com/swedfr/memorizingSAM公开发布。

英文摘要:

Segment Anything Models (SAMs) have gained increasing attention in medical image analysis due to their zero-shot generalization capability in segmenting objects of unseen classes and domains when provided with appropriate user prompts. Addressing this performance gap is important to fully leverage the pre-trained weights of SAMs, particularly in the domain of volumetric medical image segmentation, where accuracy is important but well-annotated 3D medical data for fine-tuning is limited. In this work, we investigate whether introducing the memory mechanism as a plug-in, specifically the ability to memorize and recall internal representations of past inputs, can improve the performance of SAM with limited computation cost. To this end, we propose Memorizing SAM, a novel 3D SAM architecture incorporating a memory Transformer as a plug-in. Unlike conventional memorizing Transformers that save the internal representation during training or inference, our Memorizing SAM utilizes existing highly accurate internal representation as the memory source to ensure the quality of memory. We evaluate the performance of Memorizing SAM in 33 categories from the TotalSegmentator dataset, which indicates that Memorizing SAM can outperform state-of-the-art 3D SAM variant i.e., FastSAM3D with an average Dice increase of 11.36% at the cost of only 4.38 millisecond increase in inference time. The source code is publicly available at https://github.com/swedfr/memorizingSAM

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑