Deep learning-based methods have achieved promising results on surgical instrument segmentation. However, the high computation cost may limit the application of deep models to time-sensitive tasks such as online surgical video analysis for robotic-assisted surgery. Moreover, current methods may still suffer from challenging conditions in surgical images such as various lighting conditions and the presence of blood. We propose a novel Multi-frame Feature Aggregation (MFFA) module to aggregate video frame features temporally and spatially in a recurrent mode. By distributing the computation load of deep feature extraction over sequential frames, we can use a lightweight encoder to reduce the computation costs at each time step. Moreover, public surgical videos usually are not labeled frame by frame, so we develop a method that can randomly synthesize a surgical frame sequence from a single labeled frame to assist network training. We demonstrate that our approach achieves superior performance to corresponding deeper segmentation models on two public surgery datasets.
Asan Medical Center（峨山医疗中心）；Johns Hopkins University（约翰斯·霍普金斯大学）；Chungnam National University School of Medicine（忠南大学医学院）
Some endoscopic surgeries require a surgeon to hold the endoscope with one hand and the surgical instruments with the other hand to perform the actual surgery with correct vision. Recent technical advances in deep learning as well as in robotics can introduce robotics to these endoscopic surgeries. This can have numerous advantages by freeing one hand of the surgeon, which will allow the surgeon to use both hands and to use more intricate and sophisticated techniques. Recently, deep learning with convolutional neural network achieves state-of-the-art results in computer vision. Therefore, the …
University of Washington (UW)（华盛顿大学）；Rochester Institute of Technology（罗切斯特理工学院）
Accurate and real-time surgical instrument segmentation is important in the endoscopic vision of robot-assisted surgery, and significant challenges are posed by frequent instrument-tissue contacts and continuous change of observation perspective. For these challenging tasks more and more deep neural networks (DNN) models are designed in recent years. We are motivated to propose a general embeddable approach to improve these current DNN segmentation models without increasing the model parameter number. Firstly, observing the limited rotation-invariance performance of DNN, we proposed the Multi-…
Zuse Institute Berlin (ZIB)（柏林祖斯研究所）；TU Darmstadt（达姆施塔特工业大学）
Purpose: Segmentation of surgical instruments in endoscopic videos is essential for automated surgical scene understanding and process modeling. However, relying on fully supervised deep learning for this task is challenging because manual annotation occupies valuable time of the clinical experts. Methods: We introduce a teacher-student learning approach that learns jointly from annotated simulation data and unlabeled real data to tackle the erroneous learning problem of the current consistency-based unsupervised domain adaptation framework. Results: Empirical results on three datasets highlig…
German Cancer Research Center (DKFZ)（德国癌症研究中心）；University of Heidelberg（海德堡大学）
Segmentation of endoscopic images is an essential processing step for computer and robotics-assisted interventions. The Robust-MIS challenge provides the largest dataset of annotated endoscopic images to date, with 5983 manually annotated images. Here we describe OR-UNet, our optimized robust residual 2D U-Net for endoscopic image segmentation. As the name implies, the network makes use of residual connections in the encoder. It is trained with the sum of Dice and cross-entropy loss and deep supervision. During training, extensive data augmentation is used to increase the robustness. In an 8-f…
Surgical tool segmentation in endoscopic images is an important problem: it is a crucial step towards full instrument pose estimation and it is used for integration of pre- and intra-operative images into the endoscopic view. While many recent approaches based on convolutional neural networks have shown great results, a key barrier to progress lies in the acquisition of a large number of manually-annotated images which is necessary for an algorithm to generalize and work well in diverse surgical scenarios. Unlike the surgical image data itself, annotations are difficult to acquire and may be o…
Tecnológico de Monterrey（蒙特雷科技大学）；University of Oxford（牛津大学）
Image-based tracking of laparoscopic instruments plays a fundamental role in computer and robotic-assisted surgeries by aiding surgeons and increasing patient safety. Computer vision contests, such as the Robust Medical Instrument Segmentation (ROBUST-MIS) Challenge, seek to encourage the development of robust models for such purposes, providing large, diverse, and annotated datasets. To date, most of the existing models for instance segmentation of medical instruments were based on two-stage detectors, which provide robust results but are nowhere near to the real-time (5 frames-per-second (fp…
ICube, University of Strasbourg, CNRS（斯特拉斯堡大学ICube研究所，法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）；Fondazione Policlinico Universitario Agostino Gemelli IRCCS（阿戈斯蒂诺·杰梅利大学综合医院基金会IRCCS）；IRCAD France（法国消化癌症研究研究所）
Out of all existing frameworks for surgical workflow analysis in endoscopic videos, action triplet recognition stands out as the only one aiming to provide truly fine-grained and comprehensive information on surgical activities. This information, presented as <instrument, verb, target> combinations, is highly challenging to be accurately identified. Triplet components can be difficult to recognize individually; in this task, it requires not only performing recognition simultaneously for all three triplet components, but also correctly establishing the data association between them. To achieve …
PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgery
University College London（伦敦大学学院）；National Hospital for Neurology and Neurosurgery（国立神经学与神经外科医院）；Centre for AI and Robotics (CAIR) HKISI, CAS（中国科学院香港创新研究院人工智能与机器人中心）；Shanghai Jiao Tong University（上海交通大学）
Surgical robots are used to perform minimally invasive surgery and alleviate much of the burden imposed on surgeons. Our group has developed a surgical robot to aid in the removal of tumors at the base of the skull via access through the nostrils. To avoid injuring the patients, a collision-avoidance algorithm that depends on having an accurate model for the poses of the instruments' shafts is used. Given that the model's parameters can change over time owing to interactions between instruments and other disturbances, the online estimation of the poses of the instrument's shaft is essential. I…
ICube（ICube实验室）；University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；University Hospital of Strasbourg（斯特拉斯堡大学医院）；IRCAD（法国消化外科研究所）；Fondazione Policlinico Universitario Agostino Gemelli IRCCS（阿戈斯蒂诺·杰梅利大学综合医院基金会IRCCS）
Recognition of surgical activity is an essential component to develop context-aware decision support for the operating room. In this work, we tackle the recognition of fine-grained activities, modeled as action triplets <instrument, verb, target> representing the tool activity. To this end, we introduce a new laparoscopic dataset, CholecT40, consisting of 40 videos from the public dataset Cholec80 in which all frames have been annotated using 128 triplet classes. Furthermore, we present an approach to recognize these triplets directly from the video data. It relies on a module called Class Act…
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）
Background. Flexible bendable instruments are key tools for performing surgical endoscopy. Being able to measure the 3D position of such instruments can be useful for various tasks, such as controlling automatically robotized instruments and analyzing motions. Methods. We propose an automatic method to infer the 3D pose of a single bending section instrument, using only the images provided by a monocular camera embedded at the tip of the endoscope. The proposed method relies on colored markers attached onto the bending section. The image of the instrument is segmented using a graph-based metho…
Endoscopic images typically contain several artifacts. The artifacts significantly impact image analysis result in computer-aided diagnosis. Convolutional neural networks (CNNs), a type of deep learning, can removes such artifacts. Various architectures have been proposed for the CNNs, and the accuracy of artifact removal varies depending on the choice of architecture. Therefore, it is necessary to determine the artifact removal accuracy, depending on the selected architecture. In this study, we focus on endoscopic surgical instruments as artifacts, and determine and discuss the artifact remov…
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院）；Politecnico di Milano（米兰理工大学）
Automatic surgical instrument segmentation of endoscopic images is a crucial building block of many computer-assistance applications for minimally invasive surgery. So far, state-of-the-art approaches completely rely on the availability of a ground-truth supervision signal, obtained via manual annotation, thus expensive to collect at large scale. In this paper, we present FUN-SIS, a Fully-UNsupervised approach for binary Surgical Instrument Segmentation. FUN-SIS trains a per-frame segmentation model on completely unlabelled endoscopic videos, by solely relying on implicit motion information an…
Large Language Model Evaluated Stand-alone Attention-Assisted Graph Neural Network with Spatial and Structural Information Interaction for Precise Endoscopic Image Segmentation
Department of Mathematics, The Chinese University of Hong Kong（香港中文大学数学系）；Department of Electronic and Computer Engineering, University of Massachusetts, Lowell（马萨诸塞大学低罗分校电子与计算机工程系）；Department of Radiology, Northwestern University（西北大学放射科）
Zuse Institute Berlin (ZIB)（柏林祖斯研究所）；TU Darmstadt（达姆施塔特工业大学）
Surgical tool segmentation in endoscopic videos is an important component of computer assisted interventions systems. Recent success of image-based solutions using fully-supervised deep learning approaches can be attributed to the collection of big labeled datasets. However, the annotation of a big dataset of real videos can be prohibitively expensive and time consuming. Computer simulations could alleviate the manual labeling problem, however, models trained on simulated data do not generalize to real data. This work proposes a consistency-based framework for joint learning of simulated and r…
SenseTime Research（商汤科技研究院）；The University of Hong Kong（香港大学）
Precise localization of polyp is crucial for early cancer screening in gastrointestinal endoscopy. Videos given by endoscopy bring both richer contextual information as well as more challenges than still images. The camera-moving situation, instead of the common camera-fixed-object-moving one, leads to significant background variation between frames. Severe internal artifacts (e.g. water flow in the human body, specular reflection by tissues) can make the quality of adjacent frames vary considerately. These factors hinder a video-based model to effectively aggregate features from neighborhood …
Robust Real-Time Endoscopic Stereo Matching under Fuzzy Tissue Boundaries
The School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China（生物医学工程学院，上海交通大学）；Innovation Center for Electronic Design Automation Technology, Hangzhou Dianzi University, Hangzhou 310018, China（电子设计自动化技术创新中心，杭州电子科技大学）
Center for Research and Formation in Artificial Intelligence, Universidad de los Andes（安第斯大学人工智能研究与培训中心）
We study the task of semantic segmentation of surgical instruments in robotic-assisted surgery scenes. We propose the Instance-based Surgical Instrument Segmentation Network (ISINet), a method that addresses this task from an instance-based segmentation perspective. Our method includes a temporal consistency module that takes into account the previously overlooked and inherent temporal information of the problem. We validate our approach on the existing benchmark for the task, the Endoscopic Vision 2017 Robotic Instrument Segmentation Dataset, and on the 2018 version of the dataset, whose anno…
Technische Universität München（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；Intuitive Surgical Inc.（直观医疗公司）
Detection, tracking, and pose estimation of surgical instruments are crucial tasks for computer assistance during minimally invasive robotic surgery. In the majority of cases, the first step is the automatic segmentation of surgical tools. Prior work has focused on binary segmentation, where the objective is to label every pixel in an image as tool or background. We improve upon previous work in two major ways. First, we leverage recent techniques such as deep residual learning and dilated convolutions to advance binary-segmentation performance. Second, we extend the approach to multi-class se…
Video streams are utilised to guide minimally-invasive surgery and diagnostic procedures in a wide range of procedures, and many computer assisted techniques have been developed to automatically analyse them. These approaches can provide additional information to the surgeon such as lesion detection, instrument navigation, or anatomy 3D shape modeling. However, the necessary image features to recognise these patterns are not always reliably detected due to the presence of irregular light patterns such as specular highlight reflections. In this paper, we aim at removing specular highlights from…
Wellcome/EPSRC Centre for Interventional and Surgical Sciences, University College London（伦敦大学学院威康/EPSRC介入与外科科学中心）；Department of Neurosurgery, The National Hospital for Neurology and Neurosurgery（国立神经病学与神经外科医院神经外科）
Video motion magnification could directly highlight subsurface blood vessels in endoscopic video in order to prevent inadvertent damage and bleeding. Applying motion filters to the full surgical image is however sensitive to residual motion from the surgical instruments and can impede practical application due to aberration motion artefacts. By storing the temporal filter response from local spatial frequency information for a single cardiovascular cycle prior to tool introduction to the scene, a filter can be used to determine if motion magnification should be active for a spatial region of t…
Massachusetts Institute of Technology（麻省理工学院）；Neuromation OU；University of Michigan（密歇根大学）；Lyft Inc.（来福公司）
Semantic segmentation of robotic instruments is an important problem for the robot-assisted surgery. One of the main challenges is to correctly detect an instrument's position for the tracking and pose estimation in the vicinity of surgical scenes. Accurate pixel-wise instrument segmentation is needed to address this challenge. In this paper we describe our winning solution for MICCAI 2017 Endoscopic Vision SubChallenge: Robotic Instrument Segmentation. Our approach demonstrates an improvement over the state-of-the-art results using several novel deep neural network architectures. It addressed…
EndoCaver: Handling Fog, Blur and Glare in Endoscopic Images via Joint Deblurring-Segmentation
1 CyPhi ( ) AI Research Lab, School of IT, Monash University, Malaysia Campus；2 Department of Electronic \& Computer Engineering, The Hong Kong University of Science；3 Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
National University of Sciences and Technology（国家科技大学）；National Center for Artificial Intelligence（国家人工智能中心）
Autonomous surgical procedures, in particular minimal invasive surgeries, are the next frontier for Artificial Intelligence research. However, the existing challenges include precise identification of the human anatomy and the surgical settings, and modeling the environment for training of an autonomous agent. To address the identification of human anatomy and the surgical settings, we propose a deep learning based semantic segmentation algorithm to identify and label the tissues and organs in the endoscopic video feed of the human torso region. We present an annotated dataset, m2caiSeg, creat…
Registering Neural 4D Gaussians for Endoscopic Surgery
The Chinese University of Hong Kong (CUHK)（香港中文大学）；Shenzhen Research Institute, CUHK（香港中文大学深圳研究院）；KTH, Kungliga Tekniska högskolan Royal Institute of Technology（瑞典皇家理工学院）；Hong Kong Applied Science and Technology Research Institute Company Limited（香港应用科技研究院有限公司）
German Cancer Research Center（德国癌症研究中心）；National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心（NCT））；University of Heidelberg（海德堡大学）
Surgical data science is a new research field that aims to observe all aspects of the patient treatment process in order to provide the right assistance at the right time. Due to the breakthrough successes of deep learning-based solutions for automatic image annotation, the availability of reference annotations for algorithm training is becoming a major bottleneck in the field. The purpose of this paper was to investigate the concept of self-supervised learning to address this issue. Our approach is guided by the hypothesis that unlabeled video data can be used to learn a representation of the…
Geo-RepNet: Geometry-Aware Representation Learning for Surgical Phase Recognition in Endoscopic Submucosal Dissection
Department of Electronic Engineering, The Chinese University of Hong Kong（电子工程系，香港中文大学）；Theory Lab, Central Research Institute, 2012 Labs, Huawei Technologies Co. Ltd.（理论实验室，中央研究院，2012实验室，华为技术有限公司）
CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissection
The Chinese University of Hong Kong（香港中文大学）；Shanghai AI Laboratory（上海人工智能实验室）；The Qilu Hospital of Shandong University（山东大学齐鲁医院）；National University o（国立大学）
Endo-TTAP: Robust Endoscopic Tissue Tracking via Multi-Facet Guided Attention and Hybrid Flow-point Supervision
College of Mechatronics and Engineering, Shenzhen University（深圳大学机电与控制工程学院）；The Chinese University of Hong Kong（香港中文大学）；CUHK Shenzhen Research Institute（香港中文大学（深圳）研究院）；Shenzhen People’s Hospital（深圳市人民医院）；College of Engineering Southern University of Science and Technology (SUSTech)（南方科技大学工程学院）
Xiamen University（厦门大学）；The Chinese University of Hong Kong（香港中文大学）；Zhongshan Hospital affiliated to Xiamen University（厦门大学附属中山医院）；The Hong Kong Polytechnic University（香港理工大学）
We propose a novel shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection (ESD) surgery. This task is of great clinical significance but extremely challenging due to bleeding, lighting reflection, and motion blur in the complicated surgical environment. Compared with existing solutions, which either neglect geometric relationships among targeting objects or capture the relationships by using complicated aggregation schemes, the proposed network is capable of achieving satisfactory accuracy while maintaining real-time performance by taking…
German Cancer Research Center（德国癌症研究中心）；University of Heidelberg（海德堡大学）；Heidelberg University Hospital（海德堡大学医院）；caresyntax；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；SimulaMet；Arctic University of Norway（挪威北极大学）；Oslo Metropolitan University（奥斯都会大学）；University of Electronic Science and Technology of China（电子科技大学）；National Center for Tumor Diseases（国家肿瘤疾病中心）；Technische Universität Dresden（德累斯顿工业大学）；Helmholtz-Zentrum Dresden - Rossendorf（德累斯顿-罗森多夫亥姆霍兹中心）；Universidad de los Andes（安第斯大学）；Peking University（北京大学）；Xiamen University（厦门大学）
Intraoperative tracking of laparoscopic instruments is often a prerequisite for computer and robotic-assisted interventions. While numerous methods for detecting, segmenting and tracking of medical instruments based on endoscopic video images have been proposed in the literature, key limitations remain to be addressed: Firstly, robustness, that is, the reliable performance of state-of-the-art methods when run on challenging images (e.g. in the presence of blood, smoke or motion artifacts). Secondly, generalization; algorithms trained for a specific intervention in a specific hospital should ge…
State-Change Learning for Prediction of Future Events in Endoscopic Videos
University of Strasbourg, CNRS, INSERM, ICube, UMR7357, France（斯特拉斯堡大学，法国国家科学研究中心，法国国家医学研究院，ICube，UMR7357，法国）；University Hospital of Strasbourg, France（斯特拉斯堡大学医院，法国）
Endo3R: Unified Online Reconstruction from Dynamic Monocular Endoscopic Video
The Chinese University of Hong Kong（香港中文大学）；Hong Kong Center for Logistics Robotics（香港物流机器人中心）；Johns Hopkins University（约翰斯·霍普金斯大学）；Shanghai Jiao Tong University（上海交通大学）
The CycleGAN framework allows for unsupervised image-to-image translation of unpaired data. In a scenario of surgical training on a physical surgical simulator, this method can be used to transform endoscopic images of phantoms into images which more closely resemble the intra-operative appearance of the same surgical target structure. This can be viewed as a novel augmented reality approach, which we coined Hyperrealism in previous work. In this use case, it is of paramount importance to display objects like needles, sutures or instruments consistent in both domains while altering the style t…
University of California San Diego（加利福尼亚大学圣地亚哥分校）；Zhejiang University（浙江大学）；Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies（阿里巴巴-浙江大学前沿技术联合研究院）
Traditional control and task automation have been successfully demonstrated in a variety of structured, controlled environments through the use of highly specialized modeled robotic systems in conjunction with multiple sensors. However, the application of autonomy in endoscopic surgery is very challenging, particularly in soft tissue work, due to the lack of high-quality images and the unpredictable, constantly deforming environment. In this work, we propose a novel surgical perception framework, SuPer, for surgical robotic control. This framework continuously collects 3D geometric information…
University of Washington（华盛顿大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Rochester Institute of Technology（罗切斯特理工学院）
Intelligent vision is appealing in computer-assisted and robotic surgeries. Vision-based analysis with deep learning usually requires large labeled datasets, but manual data labeling is expensive and time-consuming in medical problems. We investigate a novel cross-domain strategy to reduce the need for manual data labeling by proposing an image-to-image translation model live-cadaver GAN (LC-GAN) based on generative adversarial networks (GANs). We consider a situation when a labeled cadaveric surgery dataset is available while the task is instrument segmentation on an unlabeled live surgery da…
CholecTriplet2022: Show me a tool and tell me the triplet -- an endoscopic vision challenge for surgical action triplet detection
University of Strasbourg（斯特拉斯堡大学）；CNRS（法国国家科学研究中心）；IHU Strasbourg（斯特拉斯堡大学医院研究所）；Technical University Munich（慕尼黑工业大学）；Riwolink GmbH（Riwolink有限公司）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；Shanghai Jiao Tong University（上海交通大学）；IPCA（葡萄牙卡瓦多和阿韦河理工学院）；University of Minho（米尼奥大学）
Comparative validation of surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation in endoscopy: Results of the PhaKIR 2024 challenge
Regensburg Medical Image Computing (ReMIC), OTH Regensburg（雷根萨大学医学影像计算中心）；Research Group MITI, TUM University Hospital, School of Medicine and Health（技术大学慕尼黑大学医院MITI研究组）；Regensburg Center of Biomedical Engineering (RCBE), OTH Regensburg（雷根萨生物医学工程研究中心）；Regensburg University（雷根萨大学）；Regensburg Center of Health Sciences and Technology (RCHST), OTH Regensburg（雷根萨健康科学与技术研究中心）；AI Centre of Excellence, Medtronic Ltd.（医学影像人工智能卓越中心）；Engineering Sciences, University College London（伦敦大学学院工程科学系）；Augmented Intelligence Lab, Kyung Hee University（庆熙大学增强智能实验室）；University of Minho, Braga（明霍大学）；Jmees Inc.（Jmees公司）；School of Sciences, São Paulo State University (UNESP), Bauru（圣保罗州立大学科学学院）；KIST HARILAB, Center for Humanoid Research, Artificial Intelligence and Robot Institute, Korea Institute of Science and Technology (KIST)（韩国科学技术院HARILAB中心）；King's College London（伦敦国王学院）；The Chinese University of Hong Kong（香港中文大学）；Division of Intelligent Medical Systems, German Cancer Research Center (DKFZ)（德国癌症研究中心智能医疗系统部门）；Muroran Institute of Technology, Hokkaido（北海道Muroran技术学院）；Niigata University of Health and Welfare（Niigata医疗福利大学）；Konica Minolta, Inc.（东宝株式会社）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）；Department of Chemical and Biological Engineering, The Hong Kong University of Science and Technology（香港科技大学化学与生物工程系）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute, Shenzhen（香港科技大学深圳-香港协同创新研究院）；Center for Research and Formation in Artificial Intelligence (CinfonIA), Los Andes University, Bogota（安第斯大学人工智能研究与培养中心）；Department of Biomedical Engineering, Medical Image Analysis Group, Eindhoven University of Technology（埃因霍温理工大学生物医学工程系）；Institute of Information Technology (ITEC), Klagenfurt University（克雷格夫大学信息技术研究所）；Center for Tactile Internet with Human-in-the-loop (CeTI), TU Dresden（德累斯顿技术大学触觉互联网中心）
Video Dataset for Surgical Phase, Keypoint, and Instrument Recognition in Laparoscopic Surgery (PhaKIR)
Regensburg Medical Image Computing (ReMIC), OTH Regensburg（雷根斯堡医学影像计算中心（ReMIC））；Regensburg Center of Biomedical Engineering (RCBE), OTH Regensburg（雷根斯堡生物医学工程中心（RCBE））；Regensburg University（雷根斯堡大学）；Regensburg Center of Health Sciences（雷根斯堡健康科学中心）；Technical University of Munich (TUM)（慕尼黑技术大学）；TUM University Hospital（慕尼黑技术大学医院）；Biomedical Image Analysis Group, Department of Computing, Imperial College London（生物医学影像分析组，计算部门，伦敦帝国学院）；Research Group MITI, TUM University Hospital, School of Medicine（MITI研究组，慕尼黑技术大学医院，医学学院）；Department of Surgery, TUM University Hospital, School of Medicine（外科部门，慕尼黑技术大学医院，医学学院）
ETSM: Automating Dissection Trajectory Suggestion and Confidence Map-Based Safety Margin Prediction for Robot-assisted Endoscopic Submucosal Dissection
The Chinese University of Hong Kong（香港中文大学）；Shenzhen Research Institute, The Chinese University of Hong Kong（香港中文大学深圳研究院）；Sun Yat-sen University（中山大学）；Qilu Hospital of Shandong University（山东大学齐鲁医院）
SurgiSR4K: A High-Resolution Endoscopic Video Dataset for Robotic-Assisted Minimally Invasive Procedures
Intuitive Surgical, Inc.（Intuitive Surgical公司）；Johns Hopkins Medicine Neurosurgery（约翰霍普金斯医学神经外科）；Johns Hopkins University Electrical and Computer Engineering（约翰霍普金斯大学电气与计算机工程）；University of British Columbia Electrical and Computer Engineering（不列颠哥伦比亚大学电气与计算机工程）；Wilford & Kate Bailey Small Animal Teaching Hospital（威尔福德与凯蒂·贝利小动物教学医院）
National Center for Tumor Diseases (NCT)（国家肿瘤疾病中心（NCT））；Intuitive Surgical（直觉外科公司）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；CHU Grenoble Alpes（格勒诺布尔阿尔卑斯大学医院）；Grenoble INP（格勒诺布尔综合理工学院）；TIMC-IMAG（TIMC-IMAG实验室）；Centre for Medical Image Computing（医学影像计算中心）；University College London（伦敦大学学院）；Department of General, Visceral and Transplant Surgery, University of Heidelberg（海德堡大学普通、内脏和移植外科系）；ARTORG Center, University of Bern（伯尔尼大学ARTORG中心）；Computer Aided Medical Procedures, Johns Hopkins University（约翰霍普金斯大学计算机辅助医疗程序）；Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院（KIT）人机工程与机器人研究所）
Intraoperative segmentation and tracking of minimally invasive instruments is a prerequisite for computer- and robotic-assisted surgery. Since additional hardware like tracking systems or the robot encoders are cumbersome and lack accuracy, surgical vision is evolving as promising techniques to segment and track the instruments using only the endoscopic images. However, what is missing so far are common image data sets for consistent evaluation and benchmarking of algorithms against each other. The paper presents a comparative validation study of different vision-based methods for instrument s…
