Objective Surgical Skills Assessment and Tool Localization: Results from the MICCAI 2021 SimSurgSkill Challenge
Intuitive, Inc.（直觉外科公司）；Muroran Institute of Technology（室兰工业大学）；University College London（伦敦大学学院）；Shandong University（山东大学）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）
Timely and effective feedback within surgical training plays a critical role in developing the skills required to perform safe and efficient surgery. Feedback from expert surgeons, while especially valuable in this regard, is challenging to acquire due to their typically busy schedules, and may be subject to biases. Formal assessment procedures like OSATS and GEARS attempt to provide objective measures of skill, but remain time-consuming. With advances in machine learning there is an opportunity for fast and objective automated feedback on technical skills. The SimSurgSkill 2021 challenge (hosted as a sub-challenge of EndoVis at MICCAI 2021) aimed to promote and foster work in this endeavor. Using virtual reality (VR) surgical tasks, competitors were tasked with localizing instruments and predicting surgical skill. Here we summarize the winning approaches and how they performed. Using this publicly available dataset and results as a springboard, future work may enable more efficient training of surgeons with advances in surgical data science. The dataset can be accessed from https://console.cloud.google.com/storage/browser/isi-simsurgskill-2021.
Intuitive, Inc.（直观医疗公司）；Konica Minolta, Inc.（柯尼卡美能达公司）；University College London（伦敦大学学院）；University of Genoa（热那亚大学）；University of Burgundy（勃艮第大学）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心）
Surgical data science is revolutionizing minimally invasive surgery by enabling context-aware applications. However, many challenges exist around surgical data (and health data, more generally) needed to develop context-aware models. This work - presented as part of the Endoscopic Vision (EndoVis) challenge at the Medical Image Computing and Computer Assisted Intervention (MICCAI) 2020 conference - seeks to explore the potential for visual domain adaptation in surgery to overcome data privacy concerns. In particular, we propose to use video from virtual reality (VR) simulations of surgical exe…
The University of Tokyo（东京大学）；RIKEN（理化学研究所）；Dalian University of Technology（大连理工大学）；AIST（产业技术综合研究所）
Automated video-based assessment of surgical skills is a promising task in assisting young surgical trainees, especially in poor-resource areas. Existing works often resort to a CNN-LSTM joint framework that models long-term relationships by LSTMs on spatially pooled short-term CNN features. However, this practice would inevitably neglect the difference among semantic concepts such as tools, tissues, and background in the spatial dimension, impeding the subsequent temporal relationship modeling. In this paper, we propose a novel skill assessment framework, Video Semantic Aggregation (ViSA), wh…
University College London (UCL)（伦敦大学学院）；King’s College London (KCL)（伦敦国王学院）；Moorfields Eye Hospital（摩尔菲尔兹眼科医院）；Institute of Ophthalmology, University College London（伦敦大学学院眼科研究所）；Wellcome/EPSRC Centre for Interventional and Surgical Sciences（惠康/英国工程与物理科学研究委员会介入与外科科学中心）；School of Biomedical Engineering & Imaging Sciences（生物医学工程与影像科学学院）
Our work proposes neural network design choices that set the state-of-the-art on a challenging public benchmark on cataract surgery, CaDIS. Our methodology achieves strong performance across three semantic segmentation tasks with increasingly granular surgical tool class sets by effectively handling class imbalance, an inherent challenge in any surgical video. We consider and evaluate two conceptually simple data oversampling methods as well as different loss functions. We show significant performance gains across network architectures and tasks especially on the rarest tool classes, thereby p…
Automated segmentation of pediatric neuroblastoma on multi-modal MRI: Results of the SPPIN challenge at MICCAI 2023
Princess Máxima Center for Pediatric Oncology（玛西玛公主儿科肿瘤中心）；University Medical Center Utrecht（乌得勒支大学医学中心）；Utrecht University（乌得勒支大学）；Osaka University（大阪大学）；Muroran Institute of Technology（室兰工业大学）；Niigata University of Health and Welfare（新潟医疗福祉大学）；AGH University of Krakow（克拉科夫AGH科技大学）；HES-SO Valais-Wallis（瑞士西部应用科学与艺术大学瓦莱分校）；University of Geneva（日内瓦大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；South China Normal University（华南师范大学）；La Fe Health Research Institute（拉费健康研究所）；University of Nebraska Medical Center（内布拉斯加大学医学中心）；Wageningen Food Safety Research（瓦赫宁根食品安全研究所）
CholecTriplet2021: A benchmark challenge for surgical action triplet recognition
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University of Chicago（芝加哥大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Hefei University of Technology（合肥工业大学）；Shanghai Jiao Tong University（上海交通大学）；Argonne National Laboratory（阿贡国家实验室）；Medtronic（美敦力）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
SurgRIPE challenge: Benchmark of Surgical Robot Instrument Pose Estimation
The Hamlyn Centre for Robotic Surgery, Imperial College London, United Kingdom；ImFusion GmbH, Munich, Germany；Technical University of Munich, Munich, Germany；Eindhoven University of Technology, Eindhoven, Netherlands；Department of Engineering Physics, Tsinghua University, Beijing, China；Department of Transdisciplinary Medicine, Seoul National University Hospital, South Korea；School of Computing, University of Leeds, United Kingdom；STORMLab, University of Leeds, United Kingdom
Point Tracking in Surgery--The 2025 Surgical Tattoos in Infrared Challenge (STIRC2025)
Intuitive Surgical Inc.（直观外科公司）；ImFusion GmbH（ImFusion有限公司）；Technical University of Munich（慕尼黑工业大学）；University of British Columbia（英属哥伦比亚大学）；Johns Hopkins University（约翰·霍普金斯大学）；Daegu Gyeongbuk Institute of Science and Technology（大邱庆北科学技术院）；Hamburg University of Technology（汉堡工业大学）；SustAInLivWork Center of Excellence（可持续生活工作卓越中心）；Graduate School of Informatics, Nagoya University（名古屋大学信息科学研究生院）；Information Technology Center, Nagoya University（名古屋大学信息技术中心）；Department of Information Science, Aichi Institute of Technology（爱知工业大学信息科学系）；Research Center for Medical Bigdata, National Institute of Informatics（国立信息学研究所医学大数据研究中心）；Department of Translational Surgical Oncology, National Center for Tumor Diseases (NCT), NCT/UCC Dresden, a partnership between DKFZ, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresde（德累斯顿国家肿瘤疾病中心（NCT）转化外科肿瘤学系，NCT/UCC德累斯顿，由德国癌症研究中心、医学院和卡尔·古斯塔夫·卡鲁斯大学医院、德累斯顿工业大学合作成立）
Intuitive Surgical SurgToolLoc and SurgVU Challenges Results: 2022-2025
Intuitive Surgical, Inc.（Intuitive Surgical公司）；Muroran Institute of Technology（Muroran理工学院）；Niigata University of Health and Welfare Fujita Health University（Niigata大学健康与福利大学 Fujita健康大学）；NVIDIA, Inc.（NVIDIA公司）；University of Tokyo（东京大学）；Keio University（Keio大学）；Shun Hing Institute of Advanced Engineering（Shun Hing先进工程研究所）；NUS NUSRI SZ（新加坡大学 NUSRI SZ）；University of Strasbourg IHU Strasbourg（斯特拉斯堡大学 IHU斯特拉斯堡）；University Medical Center Hambrug-Eppendorf（汉堡-埃彭多夫大学医学中心）
SegCol Challenge: Semantic Segmentation for Tools and Fold Edges in Colonoscopy data
UCL Hawkes Institute, University College London（伦敦大学学院霍克斯学院）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；Medtronic（美敦力）；Department of Computer Science, University Liverpool（利物浦大学计算机科学系）
Surgical scene understanding and multi-tasking learning are crucial for image-guided robotic surgery. Training a real-time robotic system for the detection and segmentation of high-resolution images provides a challenging problem with the limited computational resource. The perception drawn can be applied in effective real-time feedback, surgical skill assessment, and human-robot collaborative surgeries to enhance surgical outcomes. For this purpose, we develop a novel end-to-end trainable real-time Multi-Task Learning (MTL) model with weight-shared encoder and task-aware detection and segment…
Point Tracking in Surgery--The 2024 Surgical Tattoos in Infrared (STIR) Challenge
Intuitive Surgical Inc.（直觉外科公司）；ImFusion GmbH（ImFusion 有限公司）；Technical University of Munich（慕尼黑工业大学）；NVIDIA（英伟达公司）；Daegu Gyeongbuk Institute of Science and Technology (DGIST)（大邱庆北科学技术院）；Czech Technical University in Prague（布拉格捷克技术大学）；Shenzhen Research Institute（深圳研究院）；The Chinese University of Hong Kong（香港中文大学）；Shenzhen University（深圳大学）；Life and Health Sciences Research Institute (ICVS)（生命与健康科学研究所）；University of Minho（米尼奥大学）；IPCA（波尔图高等理工学院）；University of California - Los Angeles (UCLA)（加利福尼亚大学洛杉矶分校）；Hefei University of Technology（合肥工业大学）；Hawkes Institute（霍克斯研究所）
Comparative validation of surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation in endoscopy: Results of the PhaKIR 2024 challenge
Regensburg Medical Image Computing (ReMIC), OTH Regensburg（雷根萨大学医学影像计算中心）；Research Group MITI, TUM University Hospital, School of Medicine and Health（技术大学慕尼黑大学医院MITI研究组）；Regensburg Center of Biomedical Engineering (RCBE), OTH Regensburg（雷根萨生物医学工程研究中心）；Regensburg University（雷根萨大学）；Regensburg Center of Health Sciences and Technology (RCHST), OTH Regensburg（雷根萨健康科学与技术研究中心）；AI Centre of Excellence, Medtronic Ltd.（医学影像人工智能卓越中心）；Engineering Sciences, University College London（伦敦大学学院工程科学系）；Augmented Intelligence Lab, Kyung Hee University（庆熙大学增强智能实验室）；University of Minho, Braga（明霍大学）；Jmees Inc.（Jmees公司）；School of Sciences, São Paulo State University (UNESP), Bauru（圣保罗州立大学科学学院）；KIST HARILAB, Center for Humanoid Research, Artificial Intelligence and Robot Institute, Korea Institute of Science and Technology (KIST)（韩国科学技术院HARILAB中心）；King's College London（伦敦国王学院）；The Chinese University of Hong Kong（香港中文大学）；Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)（德国癌症研究中心智能医疗系统部门）；Muroran Institute of Technology, Hokkaido（北海道Muroran技术学院）；Niigata University of Health and Welfare（Niigata医疗福利大学）；Konica Minolta, Inc.（东宝株式会社）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology（香港科技大学化学与生物工程系）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute, Shenzhen（香港科技大学深圳-香港协同创新研究院）；Center for Research and Formation in Artificial Intelligence (CinfonIA), Los Andes University, Bogota（安第斯大学人工智能研究与培养中心）；Department of Biomedical Engineering, Medical Image Analysis Group, Eindhoven University of Technology（埃因霍温理工大学生物医学工程系）；Institute of Information Technology (ITEC), Klagenfurt University（克雷格夫大学信息技术研究所）；Center for Tactile Internet with Human-in-the-loop (CeTI), TU Dresden（德累斯顿技术大学触觉互联网中心）
CAT-ViL: Co-Attention Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London（伦敦大学学院惠康/工程与物理科学研究委员会介入与外科科学中心（WEISS））；The Chinese University of Hong Kong (CUHK)（香港中文大学（CUHK））；Shun Hing Institute of Advanced Engineering, CUHK（香港中文大学信兴高等工程研究所）
The CycleGAN framework allows for unsupervised image-to-image translation of unpaired data. In a scenario of surgical training on a physical surgical simulator, this method can be used to transform endoscopic images of phantoms into images which more closely resemble the intra-operative appearance of the same surgical target structure. This can be viewed as a novel augmented reality approach, which we coined Hyperrealism in previous work. In this use case, it is of paramount importance to display objects like needles, sutures or instruments consistent in both domains while altering the style t…
Digital Surgery Ltd, Medtronic（美敦力数字外科有限公司）；ICube, University of Strasbourg, CNRS, IHU Strasbourg（斯特拉斯堡大学ICube实验室、法国国家科学研究中心、斯特拉斯堡大学医院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Hutom（Hutom公司）；Xiamen University（厦门大学）；University of Electronic Science and Technology of China（电子科技大学）；National University of Singapore（新加坡国立大学）；National Institute of Technology, Raipur（赖布尔国家理工学院）；King’s College London（伦敦国王学院）；University College London（伦敦大学学院）
Surgical scene segmentation is essential for anatomy and instrument localization which can be further used to assess tissue-instrument interactions during a surgical procedure. In 2017, the Challenge on Automatic Tool Annotation for cataRACT Surgery (CATARACTS) released 50 cataract surgery videos accompanied by instrument usage annotations. These annotations included frame-level instrument presence information. In 2020, we released pixel-wise semantic annotations for anatomy and instruments for 4670 images sampled from 25 videos of the CATARACTS training set. The 2020 CATARACTS Semantic Segmen…
OSS: Open Suturing Skills Vision-Based Assessment Challenge 2024-2025
Department of Translational Surgical Oncology, National Center for Tumor Diseases (NCT/UCC) Dresden（转化外科肿瘤学部，肿瘤疾病国家中心（NCT/UCC）德累斯顿）；The Centre for Tactile Internet with Human-in-the-Loop (CeTI), TUD Dresden University of Technology（具有人环路触觉互联网中心（CeTI），德累斯顿技术大学）；Department of Oral and Maxillofacial Surgery, University Hospital RWTH Aachen（口腔和颌面外科部，亚琛大学医院）；Center for Tooth-, Mouth- and Jaw Medicine, University Göttingen（牙科、口科和颌科医学中心，哥廷根大学）；Institute of Medical Informatics, University Hospital RWTH Aachen（医学信息学研究所，亚琛大学医院）；Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden University of Technology（医学系和卡尔·戈斯塔·卡鲁斯大学医院，德累斯顿技术大学）；German Cancer Research Center (DKFZ)（德国癌症研究中心（DKFZ））；Muroran Institute of Technology（牟然技术学院）；Niigata University of Health and Welfare（北九州市保健福利大学）；Konica Minolta, Inc.（柯尼卡美能达公司）；Jmees, Inc.（Jmees公司）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程部，香港科学与技术大学）；Center Algoritmi/LASI, University of Minho（算法中心/ALASI，米尼奥大学）；Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho（生命与健康科学研究院（ICVS），医学院，米尼奥大学）；ICVS/3B's - PT Government Associate Laboratory（ICVS/3B's - PT政府附属实验室）；Institute for AI in Medicine (IKIM), University Medicine Essen（医学人工智能研究所（IKIM），埃森大学医学部）；The Faculty of Data and Decisions Science, Technion - Israel Institute of Technology（数据与决策科学系，技术学院-以色列理工学院）；UCL Hawkes Institute, University College London（UCL Hawkes研究所，伦敦大学学院）；School of Computing, Queen's University（计算学院，皇后大学）；Department of Transdisciplinary Medicine, Seoul National University Hospital（跨学科医学部，首尔国立大学医院）；Interdisciplinary Program in Medical Informatics, Seoul National University（医学信息学跨学科项目，首尔国立大学）；Department of Clinical Medical Sciences, Seoul National University（临床医学科学部，首尔国立大学）；Institute of Convergence Medicine with Innovative Technology, Seoul National University Hospital（融合医学与创新技术研究所，首尔国立大学医院）；Department of Surgery, Seoul National University College of Medicine and Seoul National University Hospital（外科部，首尔国立大学医学院和首尔国立大学医院）
