SurGo-R1: Benchmarking and Modeling Contextual Reasoning for Operative Zone in Surgical Video
National University of Singapore, Singapore（新加坡国立大学）；Southern Medical University, China（南方医科大学）；Guangzhou Research Translation and Innovation Institute, National University of Singapore, China（广州研究翻译与创新研究院，新加坡国立大学，中国）
Guanyi Qin, Xiaozhen Wang, Zhu Zhuo, Chang Han Low, Yuancan Xiao, Yibing Fu, Haofeng Liu, Kai Wang, Chunjiang Li, Yueming Jin
Minimally invasive surgery has dramatically improved patient operative outcomes, yet identifying safe operative zones remains challenging in critical phases, requiring surgeons to integrate visual cues, procedural phase, and anatomical context under high cognitive load. Existing AI systems offer binary safety verification or static detection, ignoring the phase-dependent nature of intraoperative reasoning. We introduce ResGo, a benchmark of laparoscopic frames annotated with Go Zone bounding boxes and clinician-authored rationales covering phase, exposure quality reasoning, next action and risk reminder. We introduce evaluation metrics that treat correct grounding under incorrect phase as failures, revealing that most vision-language models cannot handle such tasks and perform poorly. We then present SurGo-R1, a model optimized via RLHF with a multi-turn phase-then-go architecture where the model first identifies the surgical phase, then generates reasoning and Go Zone coordinates conditioned on that context. On unseen procedures, SurGo-R1 achieves 76.6% phase accuracy, 32.7 mIoU, and 54.8% hardcore accuracy, a 6.6$\times$ improvement over the mainstream generalist VLMs. Code, model and benchmark will be available at https://github.com/jinlab-imvr/SurGo-R1
SurgOnAir: Hierarchy-Aware Real-Time Surgical Video Commentary
Computer Aided Medical Procedures (CAMP), TU Munich, Germany Munich Center for Machine Learning (MCML), Munich, Germany University of Strasbourg, France The Chinese University of Hong Kong, Hong Kong
GroundedSurg: A Multi-Procedure Benchmark for Language-Conditioned Surgical Tool Segmentation
King Abdullah University of Science and Technology (KAUST)（卡奥尔大学科学与技术学院）；Thapar Institute of Engineering and Technology（塔帕尔工程与技术学院）；The University of Queensland（昆士兰大学）；Gaash Research Lab, National Institute of Technology Srinagar（加什研究实验室，锡纳加尔国家理工学院）
SurgAnt-ViVQA: Learning to Anticipate Surgical Events through GRU-Driven Temporal Cross-Attention
UCL Hawkes Institute（UCL哈维斯研究所）；University College London（伦敦大学学院）；Dept of Medical Physics & Biomedical Engineering（医学物理与生物医学工程系）；UCL（伦敦大学学院）；Dept of Computer Science（计算机科学系）；National Hospital for Neurology and Neurosurgery（神经病学与神经外科国家医院）；Division of Informatics, Imaging and Data Science（信息学、成像与数据科学 division）
PitVQA: Image-grounded Text Embedding LLM for Visual Question Answering in Pituitary Surgery
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS)（惠康/工程与物理科学研究理事会介入与外科科学中心（WEISS））；University College London（伦敦大学学院）；The Chinese University of Hong Kong（香港中文大学）；National Hospital for Neurology and Neurosurgery（国家神经学与神经外科医院）
Deployment and Evaluation of an EHR-integrated, Large Language Model-Powered Tool to Triage Surgical Patients
Division of Hospital Medicine, Department of Medicine, Stanford University（斯坦福大学医院医学部，医学部）；Stanford University School of Medicine（斯坦福大学医学院）；Stanford Health Care, Technology and Digital Solutions（斯坦福健康，技术与数字解决方案）；Division of Computational Medicine, Stanford University（斯坦福大学计算医学部）；Stanford Health Care, Nursing Informatics（斯坦福健康，护理信息学）
Five billion people in the world lack access to quality surgical care. Surgeon skill varies dramatically, and many surgical patients suffer complications and avoidable harm. Improving surgical training and feedback would help to reduce the rate of complications, half of which have been shown to be preventable. To do this, it is essential to assess operative skill, a process that currently requires experts and is manual, time consuming, and subjective. In this work, we introduce an approach to automatically assess surgeon performance by tracking and analyzing tool movements in surgical videos, …
From Pre- to Intra-operative MRI: Predicting Brain Shift in Temporal Lobe Resection for Epilepsy Surgery
Surgical and Interventional Engineering（外科与介入工程）；King’s College London（伦敦国王学院）；National Hospital for Neurology and Neurosurgery（神经外科医院）；NHS（国家健康服务）；Neurosurgery, King’s College Hospital（神经外科，伦敦国王学院医院）；Fondazione IRCCS Ca’ Granda（IRCCS Ca’ Granda基金会）；Ospedale Maggiore di Milano（米兰大医院）
Computer Science and Artificial Intelligence Laboratory（计算机科学与人工智能实验室）；Massachusetts General Hospital（麻省总医院）；Toyota Research Institute（丰田研究院）；Olympus Corporation（奥林巴斯公司）
Analyzing surgical workflow is crucial for surgical assistance robots to understand surgeries. With the understanding of the complete surgical workflow, the robots are able to assist the surgeons in intra-operative events, such as by giving a warning when the surgeon is entering specific keys or high-risk phases. Deep learning techniques have recently been widely applied to recognizing surgical workflows. Many of the existing temporal neural network models are limited in their capability to handle long-term dependencies in the data, instead, relying upon the strong performance of the underlyin…
Vanderbilt University（范德堡大学）；Vanderbilt University Medical Center（范德堡大学医学中心）
Deep brain stimulation (DBS) has the potential to improve the quality of life of people with a variety of neurological diseases. A key challenge in DBS is in the placement of a stimulation electrode in the anatomical location that maximizes efficacy and minimizes side effects. Pre-operative localization of the optimal stimulation zone can reduce surgical times and morbidity. Current methods of producing efficacy probability maps follow an anatomical guidance on magnetic resonance imaging (MRI) to identify the areas with the highest efficacy in a population. In this work, we propose to revisit …
A spatio-temporal network for video semantic segmentation in surgical videos
Medtronic Digital Surgery（美敦力数字外科）；Wellcome/EPSRC Centre for Internventional and Surgical Sciences, University College London（伦敦大学学院惠康/工程与物理科学研究委员会介入与外科学中心）
Bournemouth University（伯恩茅斯大学）；Technical University of Munich（慕尼黑工业大学）
Automatic surgical instruction generation is a prerequisite towards intra-operative context-aware surgical assistance. However, generating instructions from surgical scenes is challenging, as it requires jointly understanding the surgical activity of current view and modelling relationships between visual information and textual description. Inspired by the neural machine translation and imaging captioning tasks in open domain, we introduce a transformer-backboned encoder-decoder network with self-critical reinforcement learning to generate instructions from surgical images. We evaluate the ef…
Early Operative Difficulty Assessment in Laparoscopic Cholecystectomy via Snapshot-Centric Video Analysis
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, France（法国斯特拉斯堡大学，CNRS，INSERM，ICube，UMR7357）；IHU Strasbourg, Strasbourg, France（斯特拉斯堡IHU）；Università degli Studi di Milano General Surgery Department（米兰大学外科部）；University of Torino（都灵大学）；Department of Medical Surgical Science and Translational Medicine, Sant’Andrea Hospital, Sapienza University of Rome（罗马Sapienza大学医学与外科科学及转化医学系，圣安德烈医院）；Fondazione Policlinico Universitario A. Gemelli IRCCS, Rome, Italy（罗马A.Gemelli大学医院研究院）；Università Cattolica del Sacro Cuore, Rome, Italy（罗马圣心大学）；University Hospital of Strasbourg, France（斯特拉斯堡大学医院）
Toward Real-Time Surgical Scene Segmentation via a Spike-Driven Video Transformer with Spike-Informed Pretraining
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳先进技术研究院，中国科学院）；University of Alberta（阿尔伯塔大学）；School of Medicine, Yale University（耶鲁大学医学院）；Southern University of Science and Technology（南方科技大学）；Nanfang Hospital Southern Medical University（南方医科大学南华医院）；School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）；Faculty of Computer Science and Control Engineering, Shenzhen University of Advanced Technology（深圳先进技术研究院计算机科学与控制工程学院）
Carl Zeiss AG（卡尔蔡司股份公司）；University of Padua（帕多瓦大学）；University of Applied Sciences Western Switzerland (HES-SO Valais)（瑞士西部应用科学大学（瓦莱州高等专业学院））；Carl Zeiss Meditec AG（卡尔蔡司医疗科技股份公司）；IMEC（比利时微电子研究中心）；University Hospitals Leuven（鲁汶大学医院）；KU Leuven（鲁汶大学）
Surgery for gliomas (intrinsic brain tumors), especially when low-grade, is challenging due to the infiltrative nature of the lesion. Currently, no real-time, intra-operative, label-free and wide-field tool is available to assist and guide the surgeon to find the relevant demarcations for these tumors. While marker-based methods exist for the high-grade glioma case, there is no convenient solution available for the low-grade case; thus, marker-free optical techniques represent an attractive option. Although RGB imaging is a standard tool in surgical microscopes, it does not contain sufficient …
Deep intra-operative illumination calibration of hyperspectral cameras
Siemens AG（西门子股份公司）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；Helmholtz Information and Data Science School for Health（亥姆霍兹健康信息与数据科学学院）；Heidelberg University（海德堡大学）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）；Heidelberg University Hospital（海德堡大学医院）；University Medical Center Mannheim（曼海姆大学医学中心）；DKFZ Hector Cancer Institute at the University Medical Center Mannheim（德国癌症研究中心赫克曼海姆大学医学中心癌症研究所）
Technische Universität München（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Automatic surgical phase recognition is a challenging and crucial task with the potential to improve patient safety and become an integral part of intra-operative decision-support systems. In this paper, we propose, for the first time in workflow analysis, a Multi-Stage Temporal Convolutional Network (MS-TCN) that performs hierarchical prediction refinement for surgical phase recognition. Causal, dilated convolutions allow for a large receptive field and online inference with smooth predictions even during ambiguous transitions. Our method is thoroughly evaluated on two datasets of laparoscopi…
Learning from Single Timestamps: Complexity Estimation in Laparoscopic Cholecystectomy
UCL Hawkes Institute, University College London（UCL哈维斯研究所，伦敦大学学院）；Medtronic plc.（美敦力公司）；Dept of Medical Physics & Biomedical Engineering, University College London（医学物理与生物医学工程系，伦敦大学学院）；Dept of Computer Science, University College London（计算机科学系，伦敦大学学院）
Bournemouth University（伯恩茅斯大学）；Communication University of China（中国传媒大学）
Automatic surgical gesture recognition is a prerequisite of intra-operative computer assistance and objective surgical skill assessment. Prior works either require additional sensors to collect kinematics data or have limitations on capturing temporal information from long and untrimmed surgical videos. To tackle these challenges, we propose a novel temporal convolutional architecture to automatically detect and segment surgical gestures with corresponding boundaries only using RGB videos. We devise our method with a symmetric dilation structure bridged by a self-attention module to encode and…
Mission Balance: Generating Under-represented Class Samples using Video Diffusion Models
NCT/UCC Dresden（德累斯顿NCT/UCC）；DKFZ Heidelberg（海德堡DKFZ）；Faculty of Medicine & University Hospital Carl Gustav Carus TU Dresden（德累斯顿技术大学医学学院及卡尔·古斯塔夫·卡尔医院）；HZDR Dresden, Germany（德累斯顿HZDR）；Department of Translational Surgical Oncology, NCT/UCC Dresden, Faculty of Medicine & University Hospital Carl Gustav Carus Germany（德累斯顿NCT/UCC医学系及卡尔·古斯塔夫·卡尔医院）；The Centre for Tactile Internet with Human-in-the-Loop (CeTI), TUD Dresden, Germany（德累斯顿TUD tactile internet中心）；Weldon School of Biomedical Engineering, Regenstrief Center for Healthcare Engineering (RCHE), Purdue University, USA（美国普渡大学生物医学工程学院及Regenstrief医疗工程中心）；Department of Biostatistics and Health Data Science, Richard M. Fairbanks School of Public Health, Indiana University, USA（美国印第安纳大学公共健康学院生物统计学与健康数据科学系）；Department of Visceral, Thoracic and Vascular Surgery, University Hospital & Faculty of Medicine Carl Gustav Carus, TUD Germany（德累斯顿技术大学 visceral、胸腔及血管外科系及卡尔·古斯塔夫·卡尔医院）
Digital Surgery LTD（数字手术有限公司）；Inserm（法国国家健康与医学研究院）
Video feedback provides a wealth of information about surgical procedures and is the main sensory cue for surgeons. Scene understanding is crucial to computer assisted interventions (CAI) and to post-operative analysis of the surgical procedure. A fundamental building block of such capabilities is the identification and localization of surgical instruments and anatomical structures through semantic segmentation. Deep learning has advanced semantic segmentation techniques in the recent years but is inherently reliant on the availability of labelled datasets for model training. This paper introd…
National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）
Intra-operative anticipation of instrument usage is a necessary component for context-aware assistance in surgery, e.g. for instrument preparation or semi-automation of robotic tasks. However, the sparsity of instrument occurrences in long videos poses a challenge. Current approaches are limited as they assume knowledge on the timing of future actions or require dense temporal segmentations during training and inference. We propose a novel learning task for anticipation of instrument usage in laparoscopic videos that overcomes these limitations. During training, only sparse instrument annotati…
National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）；University Hospital Carl Gustav Carus（卡尔·古斯塔夫·卡鲁斯大学医院）；TU Dresden（德累斯顿工业大学）；Centre for Tactile Internet with Human-in-the-Loop (CeTI)（人在环触觉互联网中心）
Automatically recognizing surgical gestures is a crucial step towards a thorough understanding of surgical skill. Possible areas of application include automatic skill assessment, intra-operative monitoring of critical surgical steps, and semi-automation of surgical tasks. Solutions that rely only on the laparoscopic video and do not require additional sensor hardware are especially attractive as they can be implemented at low cost in many scenarios. However, surgical gesture recognition based only on video is a challenging problem that requires effective means to extract both visual and tempo…
Weakly Supervised Temporal Convolutional Networks for Fine-grained Surgical Activity Recognition
University of Verona（维罗纳大学）；University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）；IRCAD（斯特拉斯堡微创外科研究中心）；Fondazione Policlinico Universitario Agostino Gemelli IRCCS（阿戈斯蒂诺·杰梅利大学医院基金会IRCCS）
The Chinese University of Hong Kong（香港中文大学）；T Stone Robotics Institute（T Stone机器人研究所）
Predicting future frames for robotic surgical video is an interesting, important yet extremely challenging problem, given that the operative tasks may have complex dynamics. Existing approaches on future prediction of natural videos were based on either deterministic models or stochastic models, including deep recurrent neural networks, optical flow, and latent space modeling. However, the potential in predicting meaningful movements of robots with dual arms in surgical scenarios has not been tapped so far, which is typically more challenging than forecasting independent motions of one arm rob…
Towards Surgical World-Action Modeling: A Preliminary Joint Visual-Trajectory Forecasting for Surgical Motion Planning
University of Macau（澳门大学）；University of Macau Advanced Research Institute in Hengqin（澳门大学横琴研究院）；The Chinese University of Hong Kong（香港中文大学）；The Hong Kong Polytechnic University（香港理工大学）；Shanghai Jiao Tong University（上海交通大学）；Tongren Hospital（同仁医院）；Duke University（杜克大学）
Free-DyGS: Camera-Pose-Free Scene Reconstruction for Dynamic Surgical Videos with Gaussian Splatting
Department of Biomedical Engineering, National University of Singapore (NUS)（生物医学工程系，新加坡国立大学）；National University of Singapore（新加坡国立大学）；Yong Loo Lin School of Medicine, NUS（Yong Loo Lin医学院，新加坡国立大学）；Department of Surgery, National University Hospital（国立大学医院外科部）；Centre for Smart Health, The Hong Kong Polytechnic University（智能健康中心，香港理工大学）
Efficient Frame Extraction: A Novel Approach Through Frame Similarity and Surgical Tool Tracking for Video Segmentation
University of Texas Southwestern Medical Center（德克萨斯大学西南医学中心）；NorthShore University HealthSystem（北 Shore 大学健康系统）；University of Pittsburgh Medical Center（匹兹堡大学医学中心）
Deep-Learning-Assisted Analysis of Cataract Surgery Videos
Alpen-Adria-Universität Klagenfurt（阿尔卑斯-亚得里亚大学克拉根福分校）；University of Applied Sciences Western Switzerland（瑞士西部应用科学大学）；University of Bern（伯尔尼大学）；University of Geneva（日内瓦大学）
