Systematic Evaluation and Guidelines for Segment Anything Model in Surgical Video Analysis
Global College（全球学院）；Shanghai Jiao Tong University（上海交通大学）；Department of Automation（自动化系）；National University of Singapore（新加坡国立大学）；Department of Computer Science and Engineering（计算机科学与工程系）
Surgical video segmentation is critical for AI to interpret spatial-temporal dynamics in surgery, yet model performance is constrained by limited annotated data. The SAM2 model, pretrained on natural videos, offers potential for zero-shot surgical segmentation, but its applicability in complex surgical environments, with challenges like tissue deformation and instrument variability, remains unexplored. We present the first comprehensive evaluation of the zero-shot capability of SAM2 in 9 surgical datasets (17 surgery types), covering laparoscopic, endoscopic, and robotic procedures. We analyze various prompting (points, boxes, mask) and {finetuning (dense, sparse) strategies}, robustness to surgical challenges, and generalization across procedures and anatomies. Key findings reveal that while SAM2 demonstrates notable zero-shot adaptability in structured scenarios (e.g., instrument segmentation, {multi-organ segmentation}, and scene segmentation), its performance varies under dynamic surgical conditions, highlighting gaps in handling temporal coherence and domain-specific artifacts. These results highlight future pathways to adaptive data-efficient solutions for the surgical data science field.
TSMS-SAM2: Multi-scale Temporal Sampling Augmentation and Memory-Splitting Pruning for Promptable Video Object Segmentation and Tracking in Surgical Scenarios
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS)（Wellcome/EPSRC介入与外科科学中心（WEISS））；University College London（伦敦大学学院）；Dept. of Computer Science, University College London（伦敦大学学院计算机科学系）
CycleSAM: Few-Shot Surgical Scene Segmentation with Cycle- and Scene-Consistent Feature Matching
University of Strasbourg, CNRS, INSERM, ICube, UMR7357（斯特拉斯堡大学、国家科学研究中心、法国国家医学研究院、ICube、UMR7357）；IHU Strasbourg（斯特拉斯堡IHU）；Fondazione Policlinico Universitario A. Gemelli IRCCS（阿·吉梅利大学附属医院基金会）
Object Tokens as a Bridge Between Segmentation and Visual Question Answering in Robotic Surgery
Department of Biomedical Engineering, Eindhoven University of Technology（埃因霍温理工大学生物医学工程系）；Department of Electrical Engineering, Eindhoven University of Technology（埃因霍温理工大学电气工程系）；Department of Surgery, University Medical Center Utrecht（乌得勒支大学医学中心外科）
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, Strasbourg, France（斯特拉斯堡大学、法国国家科学研究中心、法国国家医学研究院、ICube、UMR7357、法国斯特拉斯堡）；Department of General Surgery, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan（高雄长庚纪念医院外科部、长庚大学医学院）
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；Fondazione Policlinico Universitario A. Gemelli IRCCS（杰梅利大学医院基金会）；IHU-Strasbourg, Institute of Image-Guided Surgery（斯特拉斯堡大学医院-图像引导手术研究所）；Institute for Research against Digestive Cancer (IRCAD)（消化癌症研究所）
Minimally invasive image-guided surgery heavily relies on vision. Deep learning models for surgical video analysis could therefore support visual tasks such as assessing the critical view of safety (CVS) in laparoscopic cholecystectomy (LC), potentially contributing to surgical safety and efficiency. However, the performance, reliability and reproducibility of such models are deeply dependent on the quality of data and annotations used in their development. Here, we present a protocol, checklists, and visual examples to promote consistent annotation of hepatocystic anatomy and CVS criteria. We…
Spectral Adapters for Segment Anything Model-based Segmentation of Colorectal Liver Metastases in Computed Tomography
Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；University of Alberta（阿尔伯塔大学）；Alberta Machine Intelligence Institute（阿尔伯塔机器智能研究所）；The University of Texas MD Anderson Cancer Center（德克萨斯大学MD安德森癌症中心）；Toronto General Hospital Research Institute, University Health Network（大学健康网络多伦多总医院研究所）
Rensselaer Polytechnic Institute（伦斯勒理工学院）；Raytheon BBN Technologies（雷神BBN科技公司）；Army Research Laboratory（陆军研究实验室）
Surgical training in medical school residency programs has followed the apprenticeship model. The learning and assessment process is inherently subjective and time-consuming. Thus, there is a need for objective methods to assess surgical skills. Here, we use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to systematically survey the literature on the use of Deep Neural Networks for automated and objective surgical skill assessment, with a focus on kinematic data as putative markers of surgical competency. There is considerable recent interest in deep…
AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）；Wilmer Eye Institute（威尔默眼科研究所）；Malone Center for Engineering in Healthcare（马龙医疗工程中心）
SurgPose: Generalisable Surgical Instrument Pose Estimation using Zero-Shot Learning and Stereo Vision
Department of Computing, Imperial College London（帝国理工学院计算系）；Hamlyn Centre for Robotic Surgery, Department of Surgery and Cancer, Imperial College London（帝国理工学院外科与癌症部门机器人手术中心）
SurgicalPart-SAM: Part-to-Whole Collaborative Prompting for Surgical Instrument Segmentation
School of Computer Science, The University of Sydney（悉尼大学计算机科学学院）；School of Software Engineering, South China University of Technology（华南理工大学软件工程学院）；Department of Data Science & AI, Monash University（莫纳什大学数据科学与人工智能系）；School of Computer Science, Northwestern Polytechnical University（西北工业大学计算机科学学院）；Department of Computer Science, University of Rochester（罗切斯特大学计算机科学系）
本文提出 SurgicalPart-SAM (SP-SAM)，通过协同提示、跨模态编码器和部分到整体自适应融合与分层解码，将器械结构知识与 SAM 集成，解决手术器械分割中结构复杂和提示不灵活的问题，以最少的可调参数实现了最先进的性能。
ASI-Seg: Audio-Driven Surgical Instrument Segmentation with Surgeon Intention Understanding
Centre for Artificial Intelligence and Robotics (CAIR), Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；Chinese University of Hong Kong（香港中文大学）
Comparative validation of surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation in endoscopy: Results of the PhaKIR 2024 challenge
Regensburg Medical Image Computing (ReMIC), OTH Regensburg（雷根萨大学医学影像计算中心）；Research Group MITI, TUM University Hospital, School of Medicine and Health（技术大学慕尼黑大学医院MITI研究组）；Regensburg Center of Biomedical Engineering (RCBE), OTH Regensburg（雷根萨生物医学工程研究中心）；Regensburg University（雷根萨大学）；Regensburg Center of Health Sciences and Technology (RCHST), OTH Regensburg（雷根萨健康科学与技术研究中心）；AI Centre of Excellence, Medtronic Ltd.（医学影像人工智能卓越中心）；Engineering Sciences, University College London（伦敦大学学院工程科学系）；Augmented Intelligence Lab, Kyung Hee University（庆熙大学增强智能实验室）；University of Minho, Braga（明霍大学）；Jmees Inc.（Jmees公司）；School of Sciences, São Paulo State University (UNESP), Bauru（圣保罗州立大学科学学院）；KIST HARILAB, Center for Humanoid Research, Artificial Intelligence and Robot Institute, Korea Institute of Science and Technology (KIST)（韩国科学技术院HARILAB中心）；King's College London（伦敦国王学院）；The Chinese University of Hong Kong（香港中文大学）；Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)（德国癌症研究中心智能医疗系统部门）；Muroran Institute of Technology, Hokkaido（北海道Muroran技术学院）；Niigata University of Health and Welfare（Niigata医疗福利大学）；Konica Minolta, Inc.（东宝株式会社）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology（香港科技大学化学与生物工程系）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute, Shenzhen（香港科技大学深圳-香港协同创新研究院）；Center for Research and Formation in Artificial Intelligence (CinfonIA), Los Andes University, Bogota（安第斯大学人工智能研究与培养中心）；Department of Biomedical Engineering, Medical Image Analysis Group, Eindhoven University of Technology（埃因霍温理工大学生物医学工程系）；Institute of Information Technology (ITEC), Klagenfurt University（克雷格夫大学信息技术研究所）；Center for Tactile Internet with Human-in-the-loop (CeTI), TU Dresden（德累斯顿技术大学触觉互联网中心）
CataractSAM-2: A Domain-Adapted Model for Anterior Segment Surgery Segmentation and Scalable Ground-Truth Annotation
Thomas Jefferson High School for Science and Technology（托马斯·杰弗逊高中科学与技术学校）；Mass Eye and Ear, Harvard Medical School（麻省眼耳研究所，哈佛医学院）；Harvard Ophthalmology AI Lab, Schepens Eye Research Institute of Mass Eye and Ear, Harvard Medical School（哈佛眼科学人工智能实验室，麻省眼耳研究所的谢普恩斯眼研究学院，哈佛医学院）
How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model
Department of Electrical and Computer Engineering, Duke University, US（杜克大学电气与计算机工程系）；Department of Biostatistics & Bioinformatics, Duke University, US（杜克大学生物统计学与生物信息学系）；Department of Computer Science, Duke University, US（杜克大学计算机科学系）；Department of Radiology, Duke University, US（杜克大学放射学系）
Intraoperative Glioma Segmentation with YOLO + SAM for Improved Accuracy in Tumor Resection
Carnegie Vanguard High School（卡内基先锋高中）；Devon Preparatory High School（德文预备高中）；Lake Travis High School（特拉维斯湖高中）；Irvington High School（欧文顿高中）；Algoverse AI Research（Algoverse AI研究机构）
MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；The Chinese University of Hong Kong（香港中文大学）；Lehigh University（理海大学）；Harvard University（哈佛大学）；The University of Georgia（佐治亚大学）
Mannheim University of Applied Science（曼海姆应用科技大学）；UT Southwestern Medical Center（德克萨斯大学西南医学中心）；DZHK (German Centre for Cardiovascular Research)（德国心血管研究中心）；German Heart Centre Munich（慕尼黑德国心脏中心）；Hannover Medical School（汉诺威医学院）；Heidelberg University Hospital（海德堡大学医院）
Planning the optimal time of intervention for pulmonary valve replacement surgery in patients with the congenital heart disease Tetralogy of Fallot (TOF) is mainly based on ventricular volume and function according to current guidelines. Both of these two biomarkers are most reliably assessed by segmentation of 3D cardiac magnetic resonance (CMR) images. In several grand challenges in the last years, U-Net architectures have shown impressive results on the provided data. However, in clinical practice, data sets are more diverse considering individual pathologies and image properties derived fr…
