Surgical scene understanding demands temporally consistent tracking of instruments and tissues. For clinical use, such tracking should generalize to new centers and procedure types, yet retraining for each of them is costly and not scalable. Interactive video object segmentation offers a way toward this generalization: the target is specified at inference by a first-frame visual prompt, so a model can generalize to unseen categories and new scenarios without retraining. However, training such a generalizable model demands spatio-temporal masklet annotations at a scale and procedural diversity that existing surgical benchmarks lack. We fill this gap with iSurg, the largest surgical segmentation benchmark to our knowledge, spanning six procedure types with over 170k frames, 410k object masks, and 2.4k masklets, including an in-house clinical dataset of four 30-minute videos. Yet tracking over such long videos remains challenging: the target needs to be re-identified among visually similar objects after long absences, while long-term memory has to judge whether each stored frame genuinely depicts the target, both demanding a stable object-level semantic identity. To this end, we propose SurgSLOT, a surgical segmentation generalist that segments any prompted target through two coupled modules built on this semantic identity: Temporal Semantic Learning learns it for re-identification, and Semantic-driven Long-term Memory reuses it to select reliable memory frames, suppressing identity drift over long procedures. On the SAM2 and SAM3 backbones, SurgSLOT reaches 81.0 and 82.8 Macro Average J&F under cross-dataset evaluation, surpassing their fine-tuned counterparts by 5.1 and 5.3 points and transferring zero-shot to an unseen procedure type and unseen object categories, with the SAM2 version running in real time at 68 FPS.
Systematic Evaluation and Guidelines for Segment Anything Model in Surgical Video Analysis
Global College(全球学院);Shanghai Jiao Tong University(上海交通大学);Department of Automation(自动化系);National University of Singapore(新加坡国立大学);Department of Computer Science and Engineering(计算机科学与工程系)
SurgTPGS: Semantic 3D Surgical Scene Understanding with Text Promptable Gaussian Splatting
The Chinese University of Hong Kong, Hong Kong SAR, China(香港中文大学);Shenzhen Research Institute, CUHK, Shenzhen, China(深圳研究学院);Technical University of Munich, Munich, Germany(慕尼黑技术大学);University of Strasbourg & IHU Strasbourg, Strasbourg, France(斯特拉斯堡大学及斯特拉斯堡IHU);University College London, London, United Kingdom(伦敦大学学院)
TSMS-SAM2: Multi-scale Temporal Sampling Augmentation and Memory-Splitting Pruning for Promptable Video Object Segmentation and Tracking in Surgical Scenarios
University of Texas Southwestern Medical Center(德克萨斯大学西南医学中心)
Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS)(Wellcome/EPSRC介入与外科科学中心(WEISS));University College London(伦敦大学学院);Dept. of Computer Science, University College London(伦敦大学学院计算机科学系)
Object Tokens as a Bridge Between Segmentation and Visual Question Answering in Robotic Surgery
Department of Biomedical Engineering, Eindhoven University of Technology(埃因霍温理工大学生物医学工程系);Department of Electrical Engineering, Eindhoven University of Technology(埃因霍温理工大学电气工程系);Department of Surgery, University Medical Center Utrecht(乌得勒支大学医学中心外科)
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, Strasbourg, France(斯特拉斯堡大学、法国国家科学研究中心、法国国家医学研究院、ICube、UMR7357、法国斯特拉斯堡);Department of General Surgery, Kaohsiung Chang Gung Memorial Hospital, Chang Gung University College of Medicine, Kaohsiung, Taiwan(高雄长庚纪念医院外科部、长庚大学医学院)
ASI-Seg: Audio-Driven Surgical Instrument Segmentation with Surgeon Intention Understanding
Centre for Artificial Intelligence and Robotics (CAIR), Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences(中国科学院香港创新研究院人工智能与机器人中心);State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室);Chinese University of Hong Kong(香港中文大学)
Spectral Adapters for Segment Anything Model-based Segmentation of Colorectal Liver Metastases in Computed Tomography
Memorial Sloan Kettering Cancer Center(纪念斯隆凯特琳癌症中心);University of Alberta(阿尔伯塔大学);Alberta Machine Intelligence Institute(阿尔伯塔机器智能研究所);The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心);Toronto General Hospital Research Institute, University Health Network(大学健康网络多伦多总医院研究所)
CataractSAM-2: A Domain-Adapted Model for Anterior Segment Surgery Segmentation and Scalable Ground-Truth Annotation
Thomas Jefferson High School for Science and Technology(托马斯·杰弗逊高中科学与技术学校);Mass Eye and Ear, Harvard Medical School(麻省眼耳研究所,哈佛医学院);Harvard Ophthalmology AI Lab, Schepens Eye Research Institute of Mass Eye and Ear, Harvard Medical School(哈佛眼科学人工智能实验室,麻省眼耳研究所的谢普恩斯眼研究学院,哈佛医学院)