Due to the inherent flexibility of prompting, foundation models have emerged as the predominant force in the fields of natural language processing and computer vision. The recent introduction of the Segment Anything Model (SAM) signifies a noteworthy expansion of the prompt-driven paradigm into the domain of image segmentation, thereby introducing a plethora of previously unexplored capabilities. However, the viability of its application to medical image segmentation remains uncertain, given the substantial distinctions between natural and medical images. In this work, we provide a comprehensive overview of recent endeavors aimed at extending the efficacy of SAM to medical image segmentation tasks, encompassing both empirical benchmarking and methodological adaptations. Additionally, we explore potential avenues for future research directions in SAM's role within medical image segmentation. While direct application of SAM to medical image segmentation does not yield satisfactory performance on multi-modal and multi-target medical datasets so far, numerous insights gleaned from these efforts serve as valuable guidance for shaping the trajectory of foundational models in the realm of medical image analysis. To support ongoing research endeavors, we maintain an active repository that contains an up-to-date paper list and a succinct summary of open-source projects at https://github.com/YichiZhang98/SAM4MIS.
Mask-Enhanced Segment Anything Model for Tumor Lesion Semantic Segmentation
Beihang University（北京航空航天大学）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；The Chinese University of Hong Kong（香港中文大学）；Sun Yat-sen University（中山大学）；Peking University Cancer Hospital & Institute（北京大学肿瘤医院暨研究所）；Nanjing University（南京大学）
Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医学研究中心）；Shanghai Jiaotong University（上海交通大学）；the Chinese University of Hong Kong（香港中文大学）
Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation
University of Oxford（牛津大学）；National University of Singapore（新加坡国立大学）；University of Alberta（阿尔伯塔大学）；A*STAR（新加坡科技研究局）；Carnegie Mellon University（卡内基梅隆大学）；Singapore Eye Research Institute（新加坡眼科研究所）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Hongkong（香港大学）
本文提出Medical SAM Adapter（Med-SA），通过轻量适配技术将医学知识融入SAM，利用SD-Trans和HyP-Adpt适配2D模型至3D医学图像，在17个任务上以仅2%参数更新超越SOTA方法。
Enhancing the Reliability of Segment Anything Model for Auto-Prompting Medical Image Segmentation with Uncertainty Rectification
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海科学人工智能研究院）
We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation. Using our efficient model in a data collection loop, we built the largest segmentation dataset to date (by far), with over 1 billion masks on 11M licensed and privacy respecting images. The model is designed and trained to be promptable, so it can transfer zero-shot to new image distributions and tasks. We evaluate its capabilities on numerous tasks and find that its zero-shot performance is impressive -- often competitive with or even superior to prior fully supervised results. We are releas…
Nanyang Technological University（南洋理工大学）；SenseTime Research（商汤科技研究院）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Cross Prompting Consistency with Segment Anything Model for Semi-supervised Medical Image Segmentation
The Chinese University of Hong Kong（香港中文大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Institute of Medical Intelligence and XR, The Chinese University of Hong Kong（香港中文大学医学智能与XR研究所）
提出 CPC-SAM，在双分支框架中通过 SAM 交叉提示和提示一致性正则化利用标注与未标注数据，显著提升半监督医学图像分割性能。
Prompting Segment Anything Model with Domain-Adaptive Prototype for Generalizable Medical Image Segmentation
National Engineering Research Center for Multimedia Software（国家多媒体软件工程技术研究中心）；Medical Artificial Intelligence Research Institute of Renmin Hospital, Wuhan University（武汉大学人民医院医学人工智能研究院）；School of Computer Science, Central China Normal University（华中师范大学计算机学院）
TASAM: Terrain-and-Aware Segment Anything Model for Temporal-Scale Remote Sensing Segmentation
University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；Wuhan University（武汉大学）；Dalian University of Technology（大连理工大学）；University of Chinese Academy of Sciences（中国科学院大学）
I-MedSAM: Implicit Medical Image Segmentation with Segment Anything
Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Institute of Software, Chinese Academy of Sciences（中国科学院软件研究所）；Beihang University（北京航空航天大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Biomedical SAM 2: Segment Anything in Biomedical Images and Videos
University of Georgia（佐治亚大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Lehigh University（理海大学）；Northeastern University（东北大学）；University of Notre Dame（圣母大学）
Segment Anything Model Can Not Segment Anything: Assessing AI Foundation Model's Generalizability in Permafrost Mapping
Arizona State University（亚利桑那州立大学）；Woodwell Climate Research Center（伍德韦尔气候研究中心）；University of Connecticut（康涅狄格大学）；U.S. Geological Survey（美国地质调查局）；University of California, Santa Barbara（加利福尼亚大学圣巴巴拉分校）；University of Illinois at Urbana-Champaign（伊利诺伊大学香槟分校）
本文评估了以 Meta SAM 为代表的视觉基础模型在永久冻土地貌分割中的表现，通过多种提示策略和微调测试其零样本能力与领域适应性，并指出其在挑战性地理空间任务中仍需改进。
MergeSAM: Unsupervised change detection of remote sensing images based on the Segment Anything Model
School of Geography and Planning, Sun Yat-sen University（中山大学地理科学与规划学院）；Unit 33269 of the PLA（中国人民解放军33269部队）；Intelligent Science & Technology Academy Limited of CASIC（中国航天科工集团智能科技研究院有限公司）
SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model
Wuhan University（武汉大学）；The University of Sydney（悉尼大学）；National Engineering Research Center of Speech and Language Information Processing（国家语音及语言信息处理工程技术研究中心）
Eye image segmentation using visual and concept prompts with Segment Anything Model 3 (SAM3)
Lund University Humanities Lab \& Dept. of Psychology, Lund University Lund Sweden；Lund University Humanities Lab Lund Sweden；Lund University Humanities Lab \& Dept. of Psychology, Lund University；Lund University Humanities Lab
