本文提出 3DSAM-adapter，通过轻量级空间适配器将 2D SAM 整体扩展至 3D 体数据，在四个肿瘤分割数据集上验证了点击提示分割的先进性能。
摘要：
尽管 Segment Anything Model（SAM）在通用语义分割上取得了令人瞩目的结果，并在日常图像上表现出强大的泛化能力，但其在医学图像分割上所展示的性能不够精确且不稳定，尤其是在处理涉及小尺寸、不规则形状和低对比度目标的肿瘤分割任务时。值得注意的是，原始 SAM 架构是为 2D 自然图像设计的，因此无法有效地从体素医学数据中提取 3D 空间信息。本文提出了一种新颖的适配方法，用于将 SAM 从 2D 迁移到 3D，以实现可提示的医学图像分割。通过整体设计的架构修改方案，我们将 SAM 转换为支持体数据输入，同时保留其大部分预训练参数以便复用。微调过程以参数高效的方式进行：大多数预训练参数保持冻结，仅引入并调整少量轻量级空间适配器。无论自然数据与医学数据之间的领域差距，以及 2D 与 3D 在空间排列上的差异如何，在自然图像上训练的 transformer 只需进行轻量级适配，即可有效捕捉体素医学图像中的空间模式。我们在四个开源肿瘤分割数据集上进行了实验；仅使用一次点击提示，我们的模型在 4 项任务中的 3 项上优于领域内最先进的医学图像分割模型，具体而言，在肾肿瘤、胰腺肿瘤和结肠癌分割上分别提升 8.25%、29.87% 和 10.11%，并在肝肿瘤分割上取得相近性能。我们还将该适配方法与现有流行适配器进行比较，并观察到其在大多数数据集上的显著性能提升。
英文摘要：
Despite that the segment anything model (SAM) achieved impressive results on general-purpose semantic segmentation with strong generalization ability on daily images, its demonstrated performance on medical image segmentation is less precise and not stable, especially when dealing with tumor segmentation tasks that involve objects of small sizes, irregular shapes, and low contrast. Notably, the original SAM architecture is designed for 2D natural images, therefore would not be able to extract the 3D spatial information from volumetric medical data effectively. In this paper, we propose a novel adaptation method for transferring SAM from 2D to 3D for promptable medical image segmentation. Through a holistically designed scheme for architecture modification, we transfer the SAM to support volumetric inputs while retaining the majority of its pre-trained parameters for reuse. The fine-tuning process is conducted in a parameter-efficient manner, wherein most of the pre-trained parameters remain frozen, and only a few lightweight spatial adapters are introduced and tuned. Regardless of the domain gap between natural and medical data and the disparity in the spatial arrangement between 2D and 3D, the transformer trained on natural images can effectively capture the spatial patterns present in volumetric medical images with only lightweight adaptations. We conduct experiments on four open-source tumor segmentation datasets, and with a single click prompt, our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation. We also compare our adaptation method with existing popular adapters, and observed significant performance improvement on most datasets.
HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensing
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；University of Iceland（冰岛大学）；Griffith University（格里菲斯大学）
TSMS-SAM2: Multi-scale Temporal Sampling Augmentation and Memory-Splitting Pruning for Promptable Video Object Segmentation and Tracking in Surgical Scenarios
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
Prompt-Driven Building Footprint Extraction in Aerial Images with Offset-Building Model
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）；City University of Hong Kong（香港城市大学）
Mask to Concept: Auto-Promptable SAM3 via Efficient Test-Time Concept Embedding Search for Few-Shot Annotation
Wuhan University of Technology（武汉理工大学）；Huazhong University of Science and Technology（华中科技大学）；Changzhou United lmaging Healthcare Surgical Technology Co. Ltd.（常州联影医疗手术技术有限公司）
提出Mask to Concept (M2C)框架，无需外部模块或重训练，仅用少量标注图像，通过可学习概念嵌入搜索和混合不确定性估计，实现SAM3在医学图像中的自动少样本标注，达到SOTA性能。
Nanyang Technological University（南洋理工大学）；SenseTime Research（商汤科技研究院）；Carnegie Mellon University（卡内基梅隆大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
SAM 3D Animal: Promptable Animal 3D Reconstruction from Images in the Wild
University of Cambridge（剑桥大学）；Southern University of Science and Technology（南方科技大学）；Tsinghua University（清华大学）；IMATI-CNR, Milan, Italy（意大利米兰IMATI-CNR研究所）
本文提出SAM 3D Animal，首个可提示的多动物3D重建框架，基于SMAL+参数化动物模型，通过关键点和掩码提示实现拥挤和遮挡场景的可靠重建，利用Herd3D数据集在多个数据集上取得最优结果。
School of EEE（电子工程系）；Nanyang Technological University（南洋理工大学）；Institute for Infocomm Research（信息通信研究所）；A*STAR；College of Computer Science（计算机科学学院）；Nankai University（南开大学）；State Key Laboratory of Public Big Data（公共大数据国家重点实验室）；Guizhou University（贵州大学）；Fudan Vision and Learning Lab（复旦大学视觉与学习实验室）
Taming SAM3 in the Wild: A Concept Bank for Open-Vocabulary Segmentation
Department of Electrical and Computer Engineering, Sungkyunkwan University（Sungkyunkwan 大学电气与计算机工程系）；School of Computing and Artificial Intelligence, Southwest Jiaotong University（西南交通大学计算机与人工智能学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Computer Science and Engineering, University of Electronic Science and Technology of China（中国电子科技大学计算机科学与工程学院）
Opportunistic Promptable Segmentation: Leveraging Routine Radiological Annotations to Guide 3D CT Lesion Segmentation
University of Wisconsin–Madison Department of Computer Sciences（威斯康星大学麦迪逊分校计算机科学系）；University of Wisconsin–Madison Department of Radiology（威斯康星大学麦迪逊分校放射学系）
Unleashing the Power of Prompt-driven Nucleus Instance Segmentation
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Engineering, Westlake University（西湖大学工学院）；University of Liverpool（利物浦大学）
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；LIGHTSPEED；The Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）
PartSAM: A Scalable Promptable Part Segmentation Model Trained on Native 3D Data
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Hong Kong University of Science and Technology（香港科技大学）；The University of Hong Kong（香港大学）；National University of Singapore（新加坡国立大学）；Lingnan University（岭南大学）；Macau University of Science and Technology（澳门科学理工学院）
Towards Interactive Lesion Segmentation in Whole-Body PET/CT with Promptable Models
German Cancer Research Center (DKFZ) Heidelberg（德国癌症研究中心（DKFZ）海德堡）；Division of Medical Image Computing（医学影像计算 division）；HIDSS4Health - Helmholtz Information and Data Science School for Health（HIDSS4Health - 欧文信息与数据科学健康学校）；Faculty of Mathematics and Computer Science（数学与计算机科学学院）；Heidelberg University（海德堡大学）；Helmholtz Imaging（海德堡成像）；Pattern Analysis and Learning Group（模式分析与学习组）；Department of Radiation Oncology（放射肿瘤科）
University of Alberta（阿尔伯塔大学）；Tencent WeChat（腾讯微信）；Northwestern University（西北大学）；SUSTech（四川大学）；Yale University（耶鲁大学）；Utrecht University（乌得勒支大学）；Dalian University of Technology（大连理工大学）；SIAT, Chinese Academy of Sciences（中科院深圳先进技术研究院）
SemiSAM+: Rethinking Semi-Supervised Medical Image Segmentation in the Era of Foundation Models
Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化研究院）；Shanghai Academy of Artificial Intelligence for Science（上海人工智能科学研究院）；Huashan Hospital, Fudan University（复旦大学附属华山医院）；Human Phenome Institute, Fudan University（复旦大学人类表型组研究院）；School of Information Science and Engineering, Lanzhou University（兰州大学信息科学与工程学院）
AuralSAM2: Enabling SAM2 Hear Through Pyramid Audio-Visual Feature Prompting
Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Australian Institute for Machine Learning, Adelaide University（阿德莱德大学人工智能研究所）；Stanford University（斯坦福大学）；University of Central Florida（佛罗里达中央大学）；University of Surrey（萨里大学）
