MedSAM2: Segment Anything in 3D Medical Images and Videos
University Health Network(多伦多大学健康网络);Vector Institute(向量研究院);Harvard Medical School(哈佛医学院);Harvard University(哈佛大学);University of Toronto(多伦多大学);Peter Munk Cardiac Centre(彼得·芒克心脏中心);AI Hub(人工智能中心)
Jun Ma, Zongxin Yang, Sumin Kim, Bihui Chen, Mohammed Baharoon, Adibvafa Fallahpour, Reza Asakereh, Hongwei Lyu, Bo Wang
提出MedSAM2,通过微调Segment Anything Model 2构建3D医学图像和视频分割基础模型,结合人在回路流程显著降低标注成本并提升分割性能。
摘要:
医学图像和视频分割是精准医学的关键任务,在为2D图像开发特定任务或模态以及通用模型方面已取得显著进展。然而,针对3D图像和视频构建通用模型并进行全面用户研究的有限。在此,我们提出MedSAM2,一个用于3D图像和视频分割的可提示分割基础模型。该模型通过在包含超过455,000个3D图像-掩码对和76,000帧的大型医学数据集上微调Segment Anything Model 2开发而成,在广泛的器官、病灶和成像模态上超越了先前模型。此外,我们实现了人在回路的流程以促进大规模数据集创建,据我们所知,这是迄今最广泛的用户研究,涉及5,000个CT病灶、3,984个肝脏MRI病灶和251,550个超声心动图视频帧的标注,证明MedSAM2能将人工成本降低超过85%。MedSAM2还集成到广泛使用的平台中,具备用于本地和云端部署的友好界面,使其成为支持研究和医疗环境中高效、可扩展和高质量分割的实用工具。
英文摘要:
Medical image and video segmentation is a critical task for precision medicine, which has witnessed considerable progress in developing task or modality-specific and generalist models for 2D images. However, there have been limited studies on building general-purpose models for 3D images and videos with comprehensive user studies. Here, we present MedSAM2, a promptable segmentation foundation model for 3D image and video segmentation. The model is developed by fine-tuning the Segment Anything Model 2 on a large medical dataset with over 455,000 3D image-mask pairs and 76,000 frames, outperforming previous models across a wide range of organs, lesions, and imaging modalities. Furthermore, we implement a human-in-the-loop pipeline to facilitate the creation of large-scale datasets resulting in, to the best of our knowledge, the most extensive user study to date, involving the annotation of 5,000 CT lesions, 3,984 liver MRI lesions, and 251,550 echocardiogram video frames, demonstrating that MedSAM2 can reduce manual costs by more than 85%. MedSAM2 is also integrated into widely used platforms with user-friendly interfaces for local and cloud deployment, making it a practical tool for supporting efficient, scalable, and high-quality segmentation in both research and healthcare environments.
MedSAM3: Delving into Segment Anything with Medical Concepts
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Southeast University(东南大学);The Hong Kong University of Science and Technology(香港科技大学)
SAM3D: Segment Anything Model in Volumetric Medical Images
University of Arkansas(阿肯色大学);University of Science, Vietnam National University(越南国家大学科学大学);John von Neumann Institute, Vietnam National University(越南国家大学约翰·冯·诺依曼研究所);West Virginia University(西弗吉尼亚大学);University of Arkansas for Medical Sciences(阿肯色医科大学)
Segment Anything in Medical Images and Videos: Benchmark and Deployment
Peter Munk Cardiac Centre, University Health Network(彼得·芒克心脏中心,大学健康网络);Vector Institute for Artificial Intelligence(向量人工智能研究所);AI Hub, University Health Network(人工智能中心,大学健康网络);University of Toronto(多伦多大学)
本文对 SAM2 在 11 种医学图像模态和视频上进行了全面基准测试,并通过微调迁移学习及 3D Slicer 插件和 Gradio API 部署,验证了其快速适应医学领域并实现高效 3D 分割的能力。
ViPSAM: Visual Prompting Medical Image Segmentation Using Segment Anything Model
Department of Artificial Intelligence, Sungkyunkwan University, Republic of Korea(人工智能系,成均馆大学,大韩民国);Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Republic of Korea(放射肿瘤科,三星医疗中心,成均馆大学医学院,大韩民国);Department of MetaBioHealth, Sungkyunkwan University, Republic of Korea(元生物健康系,成均馆大学,大韩民国)
Peter Munk Cardiac Centre, University Health Network(彼得·芒克心脏中心,大学健康网络);University of Toronto(多伦多大学);Vector Institute(矢量研究所);Western University(西安大略大学);Tandon School of Engineering, New York University(纽约大学坦登工程学院);Yale University(耶鲁大学);UHN AI Hub(大学健康网络人工智能中心)
EviPrompt: A Training-Free Evidential Prompt Generation Method for Segment Anything Model in Medical Images
Beijing University of Posts and Telecommunications(北京邮电大学);National Institute of Health Data Science, Peking University(北京大学健康数据科学研究所);The Chinese University of Hong Kong(香港中文大学)
SQA-SAM: Segmentation Quality Assessment for Medical Images Utilizing the Segment Anything Model
Nanjing University of Science and Technology(南京理工大学);Digital Medical Research Center, School of Basic Medical Sciences, Fudan University(复旦大学基础医学院数字医学研究中心);The Chinese University of Hong Kong(香港中文大学);University of Notre Dame(圣母大学)
Adapting Segment Anything Model 3 for Concept-Driven Lesion Segmentation in Medical Images: An Experimental Study
The Medical Artificial Intelligence and Automation (MAIA) Laboratory, Department of Radiation Oncology, University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心放射肿瘤学系医学人工智能与自动化(MAIA)实验室);Medical Image Processing Group, Department of Radiology, University of Pennsylvania(宾夕法尼亚大学放射学系医学图像处理组);Division of Digestive and Liver Diseases, Department of Internal Medicine, University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心内科学系消化与肝脏疾病科)
Segment Anything Model (SAM) for Radiation Oncology
Mayo Clinic(梅奥诊所);University of Georgia(佐治亚大学);The University of Texas at Arlington(德克萨斯大学阿灵顿分校);Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院)
Encoding Structural Constraints into Segment Anything Models via Probabilistic Graphical Models
The George Washington University(乔治·华盛顿大学);Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences(深圳先进技术研究所,中国科学院);University of Alabama at Birmingham(阿拉巴马大学伯明翰分校)
Part-aware Prompted Segment Anything Model for Adaptive Segmentation
Department of Computer Science and Engineering, University of Michigan(计算机科学与工程系,密歇根大学);Department of Electrical and Computer Engineering, University of Michigan(电气与计算机工程系,密歇根大学)
Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);Digital Medical Research Center, School of Basic Medical Sciences, Fudan University(复旦大学基础医学院数字医学研究中心);Shanghai Jiaotong University(上海交通大学);the Chinese University of Hong Kong(香港中文大学)
School of Computer Science, Fudan University(复旦大学计算机科学学院);Institute of Digestive Disease, School of Medicine, Tongji University(同济大学医学院消化病研究所);National Engineering Center for Biochip(国家生物芯片工程研究中心);Endoscopy Center of Zhongshan Hospital, Fudan University(复旦大学中山医院内镜中心)
Mask-Enhanced Segment Anything Model for Tumor Lesion Semantic Segmentation
Beihang University(北京航空航天大学);Hangzhou Innovation Institute, Beihang University(北京航空航天大学杭州创新研究院);Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);The Chinese University of Hong Kong(香港中文大学);Sun Yat-sen University(中山大学);Peking University Cancer Hospital & Institute(北京大学肿瘤医院暨研究所);Nanjing University(南京大学)
Zhejiang University(浙江大学);Westlake University(西湖大学);University of Electronic Science and Technology of China(电子科技大学);Pennsylvania State University(宾夕法尼亚州立大学)