基于深度可分离扩张卷积适配器的医学目标跟踪与分割:使用Segment Anything Model 2
Depthwise-Dilated Convolutional Adapters for Medical Object Tracking and Segmentation Using the Segment Anything Model 2
AI总结:
提出DD-SAM2框架,通过深度可分离扩张适配器高效微调SAM2,实现医学视频目标跟踪与分割,在肿瘤分割和左心室跟踪上取得高Dice分数。
摘要:
近期医学图像分割的进展主要由深度学习驱动;然而,现有大多数方法仍受限于特定模态的设计,并且对动态医学影像场景的适应性较差。Segment Anything Model 2(SAM2)及其相关变体引入了用于实时视频分割的流式记忆机制,为基于提示的、可泛化的解决方案提供了新机遇。尽管如此,将这些模型适应于医学视频场景通常需要大规模数据集进行重新训练或迁移学习,导致高昂的计算成本和灾难性遗忘的风险。为应对这些挑战,我们提出了DD-SAM2,一个高效的SAM2适应框架,该框架集成了深度可分离扩张适配器(DD-Adapter),以最小参数开销增强多尺度特征提取。这种设计使得SAM2能够在有限训练数据下对医学视频进行有效微调。与现有仅关注静态图像的基于适配器的方法不同,DD-SAM2充分利用SAM2的流式记忆进行医学视频目标跟踪与分割。在TrackRad2025(肿瘤分割)和EchoNet-Dynamic(左心室跟踪)数据集上的综合评估展示了优越性能,分别取得了0.93和0.97的Dice分数。据我们所知,这项工作首次系统地探索了基于适配器的SAM2微调用于医学视频分割与跟踪。代码、数据集和模型将在https://github.com/apple1986/DD-SAM2公开提供。
英文摘要:
Recent advances in medical image segmentation have been driven by deep learning; however, most existing methods remain limited by modality-specific designs and exhibit poor adaptability to dynamic medical imaging scenarios. The Segment Anything Model 2 (SAM2) and its related variants, which introduce a streaming memory mechanism for real-time video segmentation, present new opportunities for prompt-based, generalizable solutions. Nevertheless, adapting these models to medical video scenarios typically requires large-scale datasets for retraining or transfer learning, leading to high computational costs and the risk of catastrophic forgetting. To address these challenges, we propose DD-SAM2, an efficient adaptation framework for SAM2 that incorporates a Depthwise-Dilated Adapter (DD-Adapter) to enhance multi-scale feature extraction with minimal parameter overhead. This design enables effective fine-tuning of SAM2 on medical videos with limited training data. Unlike existing adapter-based methods focused solely on static images, DD-SAM2 fully exploits SAM2's streaming memory for medical video object tracking and segmentation. Comprehensive evaluations on TrackRad2025 (tumor segmentation) and EchoNet-Dynamic (left ventricle tracking) datasets demonstrate superior performance, achieving Dice scores of 0.93 and 0.97, respectively. To the best of our knowledge, this work provides an initial attempt at systematically exploring adapter-based SAM2 fine-tuning for medical video segmentation and tracking. Code, datasets, and models will be publicly available at https://github.com/apple1986/DD-SAM2.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。