手术视频分割是计算机辅助手术中的关键任务，对于提升手术质量和患者预后至关重要。近年来，Segment Anything Model 2（SAM2）框架在图像和视频分割方面均展现出显著进展。然而，SAM2固有的贪婪选择记忆设计局限性，在手术视频的独特特性——器械快速移动、频繁遮挡以及复杂的器械-组织交互——作用下被进一步放大，导致其在复杂长视频分割中的性能下降。为应对这些挑战，我们提出了Memory Augmented（MA）-SAM2，这是一种免训练的视频目标分割策略，具备新颖的上下文感知和抗遮挡记忆模型。MA-SAM2对复杂器械运动引起的遮挡和交互表现出强鲁棒性，同时在整段视频中保持目标分割的准确性。采用多目标、单循环、单提示的推理方式进一步提升了多器械视频中跟踪过程的效率。在不引入任何额外参数或需要进一步训练的情况下，MA-SAM2在EndoVis2017和EndoVis2018数据集上分别较SAM2取得了4.36%和6.1%的性能提升，展示了其在手术实际应用中的潜力。
英文摘要：
Surgical video segmentation is a critical task in computer-assisted surgery, essential for enhancing surgical quality and patient outcomes. Recently, the Segment Anything Model 2 (SAM2) framework has demonstrated remarkable advancements in both image and video segmentation. However, the inherent limitations of SAM2's greedy selection memory design are amplified by the unique properties of surgical videos-rapid instrument movement, frequent occlusion, and complex instrument-tissue interaction-resulting in diminished performance in the segmentation of complex, long videos. To address these challenges, we introduce Memory Augmented (MA)-SAM2, a training-free video object segmentation strategy, featuring novel context-aware and occlusion-resilient memory models. MA-SAM2 exhibits strong robustness against occlusions and interactions arising from complex instrument movements while maintaining accuracy in segmenting objects throughout videos. Employing a multi-target, single-loop, one-prompt inference further enhances the efficiency of the tracking process in multi-instrument videos. Without introducing any additional parameters or requiring further training, MA-SAM2 achieved performance improvements of 4.36% and 6.1% over SAM2 on the EndoVis2017 and EndoVis2018 datasets, respectively, demonstrating its potential for practical surgical applications.
RASNet: Segmentation for Tracking Surgical Instruments in Surgical Videos Using Refined Attention Segmentation Network
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
Rethinking Surgical Instrument Segmentation: A Background Image Can Be All You Need
The Chinese University of Hong Kong（香港中文大学）；Imperial College London（帝国理工学院）；National University of Singapore（新加坡国立大学）；Shun Hing Institute of Advanced Engineering（信兴高等工程研究所）
U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument
Rochester Institute of Technology（罗切斯特理工学院）；Center for Imaging Science（影像科学中心）；Biomedical Engineering（生物医学工程）
传统的治疗方法由于有限的视野而限制了外科医生的灵活性控制。随着机器人辅助手术的出现，微创手术的医疗技术发生了范式转变。然而，在手术场景中跟踪手术器械的位置是非常具有挑战性的，并且手术工具的准确检测和识别是至关重要的。手术视频帧中基于深度学习的语义分割有可能促进这一任务。在这项工作中，我们修改了名为U-NetPlus的U-Net架构，通过引入预训练的编码器并重新设计解码器部分，通过使用基于最近邻（NN）插值的上采样操作替换转置卷积操作。为了进一步提高性能，我们还采用了非常快速和灵活的数据增强技术。我们在8 x 225帧机器人手术视频序列上训练了框架，这些视频可通过MICCAI 2017 EndoVis Challenge数据集获得，并在8 x 75帧和2 x 300帧视频上进行了测试。使用我们的U-NetPlus架构，我们报告了90.20%的DICE用于二进制分割，76.26%的DICE用于仪器部件分割，46.07%用于仪器类型（即，所有仪器）分割，优于在这些数据上实施和测试的先前技术的结果。
Unsupervised Surgical Instrument Segmentation via Anchor Generation and Semantic Diffusion
NELVT, Department of Computer Science, Peking University（北京大学计算机科学技术系数字视频编解码技术国家工程实验室）；Deepwise AI Lab（深睿医疗人工智能实验室）；Center on Frontiers of Computing Studies, Peking University（北京大学前沿计算研究中心）；Advanced Institute of Information Technology, Peking University（北京大学信息技术高等研究院）；Peking University Cancer Hospital（北京大学肿瘤医院）
A spatio-temporal network for video semantic segmentation in surgical videos
Medtronic Digital Surgery（美敦力数字外科）；Wellcome/EPSRC Centre for Internventional and Surgical Sciences, University College London（伦敦大学学院惠康/工程与物理科学研究委员会介入与外科学中心）
Systematic Evaluation and Guidelines for Segment Anything Model in Surgical Video Analysis
Global College（全球学院）；Shanghai Jiao Tong University（上海交通大学）；Department of Automation（自动化系）；National University of Singapore（新加坡国立大学）；Department of Computer Science and Engineering（计算机科学与工程系）
Surgical-DeSAM: Decoupling SAM for Instrument Segmentation in Robotic Surgery
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS)（Wellcome/EPSRC介入与外科科学中心（WEISS））；University College London（伦敦大学学院）；Dept. of Computer Science, University College London（伦敦大学学院计算机科学系）
RAUNet: Residual Attention U-Net for Semantic Segmentation of Cataract Surgical Instruments
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；CAS Center for Excellence in Brain Science and Intelligence Technology（中国科学院脑科学与智能技术卓越创新中心）
Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Video
The Chinese University of Hong Kong（香港中文大学）；Shun Hing Institute of Advanced Engineering, CUHK（香港中文大学晨兴先进工程研究所）；T Stone Robotics Institute, CUHK（香港中文大学T Stone机器人研究所）
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining
Shanghai AI Laboratory（上海人工智能实验室）；University of Strasbourg（斯特拉斯堡大学）；Monash University（莫纳什大学）；University of Michigan（密歇根大学）；Shanghai Jiao Tong University（上海交通大学）；Xiamen University（厦门大学）；The Hong Kong Polytechnic University（香港理工大学）；Technische Universität München（慕尼黑工业大学）
HieraSurg: Hierarchy-Aware Diffusion Model for Surgical Video Generation
Chair for Computer Aided Medical Procedures (CAMP), TU Munich, Germany（计算机辅助医疗程序研究所（CAMP），慕尼黑技术大学，德国）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Pseudo-label Guided Cross-video Pixel Contrast for Robotic Surgical Scene Segmentation with Limited Annotations
The Chinese University of Hong Kong（香港中文大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University College London（伦敦大学学院）；Zhejiang Lab（之江实验室）
Toward Real-Time Surgical Scene Segmentation via a Spike-Driven Video Transformer with Spike-Informed Pretraining
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；University of Alberta（阿尔伯塔大学）；School of Medicine, Yale University（耶鲁大学医学院）；Southern University of Science and Technology（南方科技大学）；Nanfang Hospital Southern Medical University（南方医科大学南华医院）；School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）；Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology（深圳先进技术研究院计算机科学与控制工程学院）
TSMS-SAM2: Multi-scale Temporal Sampling Augmentation and Memory-Splitting Pruning for Promptable Video Object Segmentation and Tracking in Surgical Scenarios
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）
ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools
University College London（伦敦大学学院）；Katholieke Universiteit Leuven（天主教鲁汶大学）；University College London Hospitals（伦敦大学学院医院）；Universitair Ziekenhuis Leuven（鲁汶大学医院）；Wellcome / EPSRC Centre for Surgical and Interventional Sciences, UCL, London, UK（惠康/英国工程与物理科学研究委员会外科与介入科学中心（伦敦大学学院））；Surgical Robot Vision Group, CMIC, Dept. Computer Science, University College London（伦敦大学学院计算机科学系CMIC外科机器人视觉小组）；Translational Imaging Group, CMIC, Dept. Med. Phys. Biomed. Eng., University College London（伦敦大学学院医学物理与生物医学工程系CMIC转化成像小组）
GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentation
King Abdullah University of Science and Technology (KAUST)（卡奥尔大学科学与技术学院）；Thapar Institute of Engineering and Technology（塔帕尔工程与技术学院）；The University of Queensland（昆士兰大学）；Gaash Research Lab, National Institute of Technology Srinagar（加什研究实验室，锡纳加尔国家理工学院）
SurgPIS: Surgical-instrument-level Instances and Part-level Semantics for Weakly-supervised Part-aware Instance Segmentation
School of Biomedical Engineering and Imaging Sciences, King’s College London（生物医学工程与成像科学学院，伦敦国王学院）；College of Electronic and Information Engineering, Tongji University（电子与信息工程学院，同济大学）
Label-supervised surgical instrument segmentation using temporal equivariance and semantic continuity
School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China（中国科学技术大学生物医学工程学院，生命科学与医学系）；Center for Medical Imaging, Robotics, Analytic Computing & Learning(MIRACLE), Suzhou Institute for Advanced Research, University of Science and Technology of China（中国科学技术大学苏州先进研究院医学影像中心、机器人与分析计算与学习中心）；Key Laboratory of Precision and Intelligent Chemistry, University of Science and Technology of China（中国科学技术大学精密与智能化学重点实验室）；Key Lab of Intelligent Information Processing of Chinese Academy of Sciences(CAS), Institute of Computing Technology, CAS（中国科学院智能信息处理重点实验室，计算技术研究所）；Faculty of Hepato-Biliary-Pancreatic Surgery, The First Medical Center, Chinese PLA General Hospital（中国人民解放军总医院肝胆胰外科系）
SurgNavAR: An Augmented Reality Surgical Navigation Framework for Optical See-Through Head Mounted Displays
Biomedical Imaging Group Rotterdam, Department of Radiology & Nuclear Medicine, Erasmus MC（鹿特丹伊拉斯姆斯大学医学中心放射与核医学系鹿特丹生物医学成像组）；Department of Oral and Maxillofacial Surgery, Erasmus MC（鹿特丹伊拉斯姆斯大学医学中心口腔颌面外科）；University of Groningen（格罗宁根大学）；Department of High Performance Compute & Visualization, SURF bv（SURF bv高性能计算与可视化部）；Department of Neurosurgery, Erasmus MC（鹿特丹伊拉斯姆斯大学医学中心神经外科）；Trauma Research Unit, Department of Surgery, Erasmus MC（鹿特丹伊拉斯姆斯大学医学中心外科创伤研究单元）
Disentangling spatio-temporal knowledge for weakly supervised object detection and segmentation in surgical video
University of Pennsylvania（宾夕法尼亚大学）；Penn Computer Assisted Surgery and Outcomes Laboratory（宾夕法尼亚计算机辅助外科与结局实验室）；Department of Surgery, University of Pennsylvania（宾夕法尼亚大学外科系）；Department of Electrical and Systems Engineering, University of Pennsylvania（宾夕法尼亚大学电气与系统工程系）；Department of Computer and Information Science, University of Pennsylvania（宾夕法尼亚大学计算机与信息科学系）
Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation
University College London（伦敦大学学院）；The Chinese University of Hong Kong（香港中文大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
Frequency Domain Enhanced U-Net for Low-Frequency Information-Rich Image Segmentation in Surgical and Deep-Sea Exploration Robots
University of Electronic Science and Technology of China（电子科技大学）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science, Peking University（北京大学计算机科学系）
