首页/论文/2011.04003

arXiv 2011.040032020-11-08eess.IVcs.CV

Real-time Surgical Environment Enhancement for Robot-Assisted Minimally Invasive Surgery Based on Super-Resolution

Imperial College London（帝国理工学院）；University of Cambridge（剑桥大学）；PAII Inc（PAII公司）

Ruoxi Wang, Dandan Zhang, Qingbiao Li, Xiao-Yun Zhou, Benny Lo

英文摘要：

In Robot-Assisted Minimally Invasive Surgery (RAMIS), a camera assistant is normally required to control the position and zooming ratio of the laparoscope, following the surgeon's instructions. However, moving the laparoscope frequently may lead to unstable and suboptimal views, while the adjustment of zooming ratio may interrupt the workflow of the surgical operation. To this end, we propose a multi-scale Generative Adversarial Network (GAN)-based video super-resolution method to construct a framework for automatic zooming ratio adjustment. It can provide automatic real-time zooming for high-quality visualization of the Region Of Interest (ROI) during the surgical operation. In the pipeline of the framework, the Kernel Correlation Filter (KCF) tracker is used for tracking the tips of the surgical tools, while the Semi-Global Block Matching (SGBM) based depth estimation and Recurrent Neural Network (RNN)-based context-awareness are developed to determine the upscaling ratio for zooming. The framework is validated with the JIGSAW dataset and Hamlyn Centre Laparoscopic/Endoscopic Video Datasets, with results demonstrating its practicability.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

使用高保真台式体模上的实时器械追踪进行内窥镜垂体手术的自动手术技能评估

Automated Surgical Skill Assessment in Endoscopic Pituitary Surgery using Real-time Instrument Tracking on a High-fidelity Bench-top Phantom

University College London（伦敦大学学院）；NVIDIA（英伟达）；King’s College London（伦敦国王学院）；National Hospital for Neurology and Neurosurgery（国立神经学与神经外科医院）

本文提出基于高保真体模的内窥镜垂体手术模拟数据集，并构建PRINTNet实时器械追踪基线模型，结合多层感知器实现87%的手术技能预测准确率，验证了自动化技能评估的可行性。

2024-09-25cs.CVeess.IV

Learning and Reasoning with the Graph Structure Representation in Robotic Surgery

National University of Singapore（新加坡国立大学）；Imperial College London（伦敦帝国学院）；Singapore General Hospital（新加坡中央医院）

Learning to infer graph representations and performing spatial reasoning in a complex surgical environment can play a vital role in surgical scene understanding in robotic surgery. For this purpose, we develop an approach to generate the scene graph and predict surgical interactions between instruments and surgical region of interest (ROI) during robot-assisted surgery. We design an attention link function and integrate with a graph parsing network to recognize the surgical interactions. To embed each node with corresponding neighbouring node features, we further incorporate SageConv into the …

2020-09-10cs.CVeess.IV

Class-Incremental Domain Adaptation with Smoothing and Calibration for Surgical Report Generation

National University of Singapore（新加坡国立大学）；NUSRI Suzhou（新加坡国立大学苏州研究院）；Singapore General Hospital（新加坡中央医院）；The Chinese University of Hong Kong（香港中文大学）；Imperial College London（伦敦帝国理工学院）

Generating surgical reports aimed at surgical scene understanding in robot-assisted surgery can contribute to documenting entry tasks and post-operative analysis. Despite the impressive outcome, the deep learning model degrades the performance when applied to different domains encountering domain shifts. In addition, there are new instruments and variations in surgical tissues appeared in robotic surgery. In this work, we propose class-incremental domain adaptation (CIDA) with a multi-layer transformer-based model to tackle the new classes and domain shift in the target domain to generate surg…

2021-07-23cs.CVeess.IV

迈向泌尿外科手术机器人视觉去雾的零样本学习

Toward Zero-Shot Learning for Visual Dehazing of Urological Surgical Robots

Department of Surgery, Shanghai Key Laboratory of Gastric Neoplasms, Shanghai Institute of Digestive Surgery, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（外科科，上海胃癌重点实验室，上海消化外科研究所，瑞金医院，上海交通大学医学院）；National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家级重点实验室，计算机科学学院，北京大学）；Department of Urology, Ruijin Hospital, Shanghai Jiaotong University School of Medicine（泌尿外科，瑞金医院，上海交通大学医学院）；School of Intelligent Science and Technology, Nanjing University（智能科学与技术学院，南京大学）

针对泌尿外科手术机器人在液体环境中因气泡雾化影响视觉感知的问题，提出无监督零样本去雾方法RSF-Dehaze及首个公开的USRobot-Dehaze数据集，显著提升了模糊区域组织的恢复效果。

2024-10-02cs.CVeess.IV

Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking

University of California San Diego（加州大学圣地亚哥分校）

Accurate and robust tracking and reconstruction of the surgical scene is a critical enabling technology toward autonomous robotic surgery. Existing algorithms for 3D perception in surgery mainly rely on geometric information, while we propose to also leverage semantic information inferred from the endoscopic video using image segmentation algorithms. In this paper, we present a novel, comprehensive surgical perception framework, Semantic-SuPer, that integrates geometric and semantic information to facilitate data association, 3D reconstruction, and tracking of endoscopic scenes, benefiting dow…

2023-02-20cs.CVeess.IV

Searching for Efficient Architecture for Instrument Segmentation in Robotic Surgery

Technische Universität München（慕尼黑工业大学）；Johns Hopkins University（约翰斯·霍普金斯大学）

Segmentation of surgical instruments is an important problem in robot-assisted surgery: it is a crucial step towards full instrument pose estimation and is directly used for masking of augmented reality overlays during surgical procedures. Most applications rely on accurate real-time segmentation of high-resolution surgical images. While previous research focused primarily on methods that deliver high accuracy segmentation masks, majority of them can not be used for real-time applications due to their computational cost. In this work, we design a light-weight and highly-efficient deep residual…

2020-07-08cs.CVeess.IV

更多相关论文收起更多论文

Segmentation of Surgical Instruments for Minimally-Invasive Robot-Assisted Procedures Using Generative Deep Neural Networks

German Aerospace Center（德国航空航天中心）

This work proves that semantic segmentation on minimally invasive surgical instruments can be improved by using training data that has been augmented through domain adaptation. The benefit of this method is twofold. Firstly, it suppresses the need of manually labeling thousands of images by transforming synthetic data into realistic-looking data. To achieve this, a CycleGAN model is used, which transforms a source dataset to approximate the domain distribution of a target dataset. Secondly, this newly generated data with perfect labels is utilized to train a semantic segmentation neural networ…

2020-06-05cs.CVcs.LGcs.RO

深度学习驱动的白内障手术中手术错误预测：从模拟到实际应用

Deep learning-enabled prediction of surgical errors during cataract surgery: from simulation to real-world application

本研究利用EyeSi模拟器视频训练深度学习模型预测白内障手术错误，通过无监督域适应迁移至实际手术，实现实时预测，AUC最高达0.820（模拟）和0.663（实际），为首个相关研究。

2025-03-28eess.IV

Dynamic Reconstruction of Deformable Soft-tissue with Stereo Scope in Minimal Invasive Surgery

University of Technology, Sydney（悉尼科技大学）；Centre for Autonomous Systems（自主系统中心）

In minimal invasive surgery, it is important to rebuild and visualize the latest deformed shape of soft-tissue surfaces to mitigate tissue damages. This paper proposes an innovative Simultaneous Localization and Mapping (SLAM) algorithm for deformable dense reconstruction of surfaces using a sequence of images from a stereoscope. We introduce a warping field based on the Embedded Deformation (ED) nodes with 3D shapes recovered from consecutive pairs of stereo images. The warping field is estimated by deforming the last updated model to the current live model. Our SLAM system can: (1) Increment…

2020-03-22cs.CVcs.ROeess.IV

Real-time Virtual Intraoperative CT for Image Guided Surgery

Rochester Institute of Technology（罗切斯特理工学院）；University of Washington（华盛顿大学）；Seattle Children’s Hospital（西雅图儿童医院）

Abstract. Purpose: This paper presents a scheme for generating virtual intraoperative CT scans in order to improve surgical completeness in Endoscopic Sinus Surgeries (ESS). Approach: The work presents three methods, the tip motion-based, the tip trajectory-based, and the instrument based, along with non-parametric smoothing and Gaussian Process Regression, for virtual intraoperative CT generation. Results: The proposed methods studied and compared on ESS performed on cadavers. Surgical results show all three methods improve the Dice Similarity Coefficients > 86%, with F-score > 92% and precis…

2021-12-05cs.CVcs.LGcs.RO

ReCal-Net: Joint Region-Channel-Wise Calibrated Network for Semantic Segmentation in Cataract Surgery Videos

Department of Information Technology, Alpen-Adria-Universität Klagenfurt（克恩滕州阿尔卑斯-亚得里亚大学信息技术系）；Department of Ophthalmology, Klinikum Klagenfurt（克拉根福医院眼科）

Semantic segmentation in surgical videos is a prerequisite for a broad range of applications towards improving surgical outcomes and surgical video analysis. However, semantic segmentation in surgical videos involves many challenges. In particular, in cataract surgery, various features of the relevant objects such as blunt edges, color and context variation, reflection, transparency, and motion blur pose a challenge for semantic segmentation. In this paper, we propose a novel convolutional module termed as \textit{ReCal} module, which can calibrate the feature maps by employing region intra-an…

2021-09-25cs.CVcs.LGeess.IV

Multitask Learning of Temporal Connectionism in Convolutional Networks using a Joint Distribution Loss Function to Simultaneously Identify Tools and Phase in Surgical Videos

Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）

Surgical workflow analysis is of importance for understanding onset and persistence of surgical phases and individual tool usage across surgery and in each phase. It is beneficial for clinical quality control and to hospital administrators for understanding surgery planning. Video acquired during surgery typically can be leveraged for this task. Currently, a combination of convolutional neural network (CNN) and recurrent neural networks (RNN) are popularly used for video analysis in general, not only being restricted to surgical videos. In this paper, we propose a multi-task learning framework…

2019-05-25cs.CVcs.LGeess.IV

Surgical-LVLM：学习适配大型视觉-语言模型以用于机器人手术中的扎根视觉问答

Surgical-LVLM: Learning to Adapt Large Vision-Language Model for Grounded Visual Question Answering in Robotic Surgery

The Chinese University of Hong Kong（香港中文大学）；Universiti Malaya（马来亚大学）；Centre for Artificial Intelligence and Robotics, HKISI-CAS（香港智能系统研究所人工智能与机器人中心）；University College London（伦敦大学学院）

本文提出面向复杂手术场景的个性化大型视觉-语言模型 Surgical-LVLM，通过 VP-LoRA 与 TIT 模块增强多模态理解和视觉定位，在多个 EndoVis 基准上取得最优性能。

2025-03-16cs.AIcs.CVcs.RO

Surgery Scene Restoration for Robot Assisted Minimally Invasive Surgery

Queensland University of Technology（昆士兰科技大学）

Minimally invasive surgery (MIS) offers several advantages including minimum tissue injury and blood loss, and quick recovery time, however, it imposes some limitations on surgeons ability. Among others such as lack of tactile or haptic feedback, poor visualization of the surgical site is one of the most acknowledged factors that exhibits several surgical drawbacks including unintentional tissue damage. To the context of robot assisted surgery, lack of frame contextual details makes vision task challenging when it comes to tracking tissue and tools, segmenting scene, and estimating pose and de…

2021-09-06cs.ROeess.IV