SAM3D: Segment Anything Model in Volumetric Medical Images
University of Arkansas（阿肯色大学）；University of Science, Vietnam National University（越南国家大学科学大学）；John von Neumann Institute, Vietnam National University（越南国家大学约翰·冯·诺依曼研究所）；West Virginia University（西弗吉尼亚大学）；University of Arkansas for Medical Sciences（阿肯色医科大学）
Nhat-Tan Bui, Dinh-Hieu Hoang, Minh-Triet Tran, Gianfranco Doretto, Donald Adjeroh, Brijesh Patel, Arabinda Choudhary, Ngan Le
Image segmentation remains a pivotal component in medical image analysis, aiding in the extraction of critical information for precise diagnostic practices. With the advent of deep learning, automated image segmentation methods have risen to prominence, showcasing exceptional proficiency in processing medical imagery. Motivated by the Segment Anything Model (SAM)-a foundational model renowned for its remarkable precision and robust generalization capabilities in segmenting 2D natural images-we introduce SAM3D, an innovative adaptation tailored for 3D volumetric medical image analysis. Unlike current SAM-based methods that segment volumetric data by converting the volume into separate 2D slices for individual analysis, our SAM3D model processes the entire 3D volume image in a unified approach. Extensive experiments are conducted on multiple medical image datasets to demonstrate that our network attains competitive results compared with other state-of-the-art methods in 3D medical segmentation tasks while being significantly efficient in terms of parameters. Code and checkpoints are available at https://github.com/UARK-AICV/SAM3D.
Beihang University（北京航空航天大学）；Hefei Innovation Research Institute, Beihang University（北京航空航天大学合肥创新研究院）；Beijing Advanced Innovation Centre for Biomedical Engineering（北京生物医学工程先进创新中心）；Beijing Advanced Innovation Centre for Big Data-Based Precision Medicine（北京大数据精准医疗先进创新中心）
Recently, deep convolutional neural networks have achieved great success for medical image segmentation. However, unlike segmentation of natural images, most medical images such as MRI and CT are volumetric data. In order to make full use of volumetric information, 3D CNNs are widely used. However, 3D CNNs suffer from higher inference time and computation cost, which hinders their further clinical applications. Additionally, with the increased number of parameters, the risk of overfitting is higher, especially for medical images where data and annotations are expensive to acquire. To issue thi…
Volumetric cell segmentation in fluorescence microscopy images is important to study a wide variety of cellular processes. Applications range from the analysis of cancer cells to behavioral studies of cells in the embryonic stage. Like in other computer vision fields, most recent methods use either large convolutional neural networks (CNNs) or vision transformer models (ViTs). Since the number of available 3D microscopy images is typically limited in applications, we take a different approach and introduce a small CNN for volumetric cell segmentation. Compared to previous CNN models for cell s…
Segment Anything in Medical Images and Videos: Benchmark and Deployment
Peter Munk Cardiac Centre, University Health Network（彼得·芒克心脏中心，大学健康网络）；Vector Institute for Artificial Intelligence（向量人工智能研究所）；AI Hub, University Health Network（人工智能中心，大学健康网络）；University of Toronto（多伦多大学）
本文对 SAM2 在 11 种医学图像模态和视频上进行了全面基准测试，并通过微调迁移学习及 3D Slicer 插件和 Gradio API 部署，验证了其快速适应医学领域并实现高效 3D 分割的能力。
Stanford University（斯坦福大学）；National Institute of Standards and Technology (NIST)（美国国家标准与技术研究院）
Segmentation of additive manufacturing (AM) defects in X-ray Computed Tomography (XCT) images is challenging, due to the poor contrast, small sizes and variation in appearance of defects. Automatic segmentation can, however, provide quality control for additive manufacturing. Over recent years, three-dimensional convolutional neural networks (3D CNNs) have performed well in the volumetric segmentation of medical images. In this work, we leverage techniques from the medical imaging domain and propose training a 3D U-Net model to automatically segment defects in XCT images of AM samples. This wo…
Segment Anything Model (SAM) for Radiation Oncology
Mayo Clinic（梅奥诊所）；University of Georgia（佐治亚大学）；The University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
SQA-SAM: Segmentation Quality Assessment for Medical Images Utilizing the Segment Anything Model
Nanjing University of Science and Technology（南京理工大学）；Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医学研究中心）；The Chinese University of Hong Kong（香港中文大学）；University of Notre Dame（圣母大学）
ViPSAM: Visual Prompting Medical Image Segmentation Using Segment Anything Model
Department of Artificial Intelligence, Sungkyunkwan University, Republic of Korea（人工智能系，成均馆大学，大韩民国）；Department of Radiation Oncology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Republic of Korea（放射肿瘤科，三星医疗中心，成均馆大学医学院，大韩民国）；Department of MetaBioHealth, Sungkyunkwan University, Republic of Korea（元生物健康系，成均馆大学，大韩民国）
Encoding Structural Constraints into Segment Anything Models via Probabilistic Graphical Models
The George Washington University（乔治·华盛顿大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究所，中国科学院）；University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）
SegVol: Universal and Interactive Volumetric Medical Image Segmentation
School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；BAAI（北京智源人工智能研究院）；The Chinese University of Hong Kong（香港中文大学）；Peking University（北京大学）
Part-aware Prompted Segment Anything Model for Adaptive Segmentation
Department of Computer Science and Engineering, University of Michigan（计算机科学与工程系，密歇根大学）；Department of Electrical and Computer Engineering, University of Michigan（电气与计算机工程系，密歇根大学）
MedSAM3: Delving into Segment Anything with Medical Concepts
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Southeast University（东南大学）；The Hong Kong University of Science and Technology（香港科技大学）
NVIDIA（英伟达）；MGH and BWH Center for Clinical Data Science（麻省总医院与布列根和妇女医院临床数据科学中心）
We propose a 4D convolutional neural network (CNN) for the segmentation of retrospective ECG-gated cardiac CT, a series of single-channel volumetric data over time. While only a small subset of volumes in the temporal sequence is annotated, we define a sparse loss function on available labels to allow the network to leverage unlabeled images during training and generate a fully segmented sequence. We investigate the accuracy of the proposed 4D network to predict temporally consistent segmentations and compare with traditional 3D segmentation approaches. We demonstrate the feasibility of the 4D…
University of Innsbruck（因斯布鲁克大学）；Medizinische Universität Innsbruck（因斯布鲁克医科大学）
Convolutional neural networks are state-of-the-art for various segmentation tasks. While for 2D images these networks are also computationally efficient, 3D convolutions have huge storage requirements and require long training time. To overcome this issue, we introduce a network structure for volumetric data without 3D convolutional layers. The main idea is to include maximum intensity projections from different directions to transform the volumetric data to a sequence of images, where each image contains information of the full data. We then apply 2D convolutions to these projection images an…
Insel Data Science Center（因塞尔数据科学中心）；Inselspital, Bern University Hospital（伯尔尼大学医院因塞尔医院）；University of Bern（伯尔尼大学）；Support Center for Advanced Neuroimaging (SCAN)（高级神经成像支持中心（SCAN））；Institute for Diagnostic and Interventional Neuroradiology（诊断与介入神经放射学研究所）
Volumetric image segmentation with convolutional neural networks (CNNs) encounters several challenges, which are specific to medical images. Among these challenges are large volumes of interest, high class imbalances, and difficulties in learning shape representations. To tackle these challenges, we propose to improve over traditional CNN-based volumetric image segmentation through point-wise classification of point clouds. The sparsity of point clouds allows processing of entire image volumes, balancing highly imbalanced segmentation problems, and explicitly learning an anatomical shape. We b…
We propose a Transformer architecture for volumetric segmentation, a challenging task that requires keeping a complex balance in encoding local and global spatial cues, and preserving information along all axes of the volume. Encoder of the proposed design benefits from self-attention mechanism to simultaneously encode local and global cues, while the decoder employs a parallel self and cross attention formulation to capture fine details for boundary refinement. Empirically, we show that the proposed design choices result in a computationally efficient model, with competitive and promising res…
Contextual Embedding Learning to Enhance 2D Networks for Volumetric Image Segmentation
Shenzhen University（深圳大学）；Shenzhen University Medical School（深圳大学医学院）；School of Biomedical Engineering（生物医学工程学院）；Huawei Cloud Computing Technologies Company Limited（华为云计算技术有限公司）；Shenzhen University General Hospital（深圳大学总医院）
Mask-Enhanced Segment Anything Model for Tumor Lesion Semantic Segmentation
Beihang University（北京航空航天大学）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；The Chinese University of Hong Kong（香港中文大学）；Sun Yat-sen University（中山大学）；Peking University Cancer Hospital & Institute（北京大学肿瘤医院暨研究所）；Nanjing University（南京大学）
In recent years, 3D convolutional neural networks have become the dominant approach for volumetric medical image segmentation. However, compared to their 2D counterparts, 3D networks introduce substantially more training parameters and higher requirement for the GPU memory. This has become a major limiting factor for designing and training 3D networks for high-resolution volumetric images. In this work, we propose a novel memory-efficient network architecture for 3D high-resolution image segmentation. The network incorporates both global and local features via a two-stage U-net-based cascaded …
Zhejiang University（浙江大学）；Westlake University（西湖大学）；University of Electronic Science and Technology of China（电子科技大学）；Pennsylvania State University（宾夕法尼亚州立大学）
Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Digital Medical Research Center, School of Basic Medical Sciences, Fudan University（复旦大学基础医学院数字医学研究中心）；Shanghai Jiaotong University（上海交通大学）；the Chinese University of Hong Kong（香港中文大学）
