SLAM assisted 3D tracking system for laparoscopic surgery
United Imaging Research Institute of Intelligent Imaging（联影智能影像研究院）；University of Michigan（密歇根大学）；United Imaging（联影医疗）；United Imaging of Surgery（联影外科研究院）
Jingwei Song, Ray Zhang, Wenwei Zhang, Hao Zhou, Maani Ghaffari
A major limitation of minimally invasive surgery is the difficulty in accurately locating the internal anatomical structures of the target organ due to the lack of tactile feedback and transparency. Augmented reality (AR) offers a promising solution to overcome this challenge. Numerous studies have shown that combining learning-based and geometric methods can achieve accurate preoperative and intraoperative data registration. This work proposes a real-time monocular 3D tracking algorithm for post-registration tasks. The ORB-SLAM2 framework is adopted and modified for prior-based 3D tracking. The primitive 3D shape is used for fast initialization of the monocular SLAM. A pseudo-segmentation strategy is employed to separate the target organ from the background for tracking purposes, and the geometric prior of the 3D shape is incorporated as an additional constraint in the pose graph. Experiments from in-vivo and ex-vivo tests demonstrate that the proposed 3D tracking system provides robust 3D tracking and effectively handles typical challenges such as fast motion, out-of-field-of-view scenarios, partial visibility, and "organ-background" relative motion.
BDIS-SLAM: A lightweight CPU-based dense stereo SLAM for surgery
United Imaging Research Institute of Intelligent Imaging（联影智能影像研究院）；University of Michigan（密歇根大学）；University of Technology Sydney（悉尼科技大学）；Imperial College London（伦敦帝国理工学院）
University of Technology, Sydney（悉尼科技大学）；Centre for Autonomous Systems（自主系统中心）
In minimal invasive surgery, it is important to rebuild and visualize the latest deformed shape of soft-tissue surfaces to mitigate tissue damages. This paper proposes an innovative Simultaneous Localization and Mapping (SLAM) algorithm for deformable dense reconstruction of surfaces using a sequence of images from a stereoscope. We introduce a warping field based on the Embedded Deformation (ED) nodes with 3D shapes recovered from consecutive pairs of stereo images. The warping field is estimated by deforming the last updated model to the current live model. Our SLAM system can: (1) Increment…
Johns Hopkins University（约翰斯·霍普金斯大学）；Children’s National Hospital（国立儿童医院）
Autonomous robotic surgery has the potential to provide efficacy, safety, and consistency independent of individual surgeons skill and experience. Autonomous soft-tissue surgery in unstructured and deformable environments is especially challenging as it necessitates intricate imaging, tissue tracking and surgical planning techniques, as well as a precise execution via highly adaptable control strategies. In the laparoscopic setting, soft-tissue surgery is even more challenging due to the need for high maneuverability and repeatability under motion and vision constraints. We demonstrate the fir…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Heidelberg University Hospital（海德堡大学医院）；Friedrich-Alexander-University Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Technical University Dresden（德累斯顿工业大学）
Recent advances in reinforcement learning (RL) have increased the promise of introducing cognitive assistance and automation to robot-assisted laparoscopic surgery (RALS). However, progress in algorithms and methods depends on the availability of standardized learning environments that represent skills relevant to RALS. We present LapGym, a framework for building RL environments for RALS that models the challenges posed by surgical tasks, and sofa_env, a diverse suite of 12 environments. Motivated by surgical training, these environments are organized into 4 tracks: Spatial Reasoning, Deformab…
University College London（伦敦大学学院）；Welcome/EPSRC Centre for Interventional and Surgical Sciences（惠康/工程与物理科学研究委员会介入与外科科学中心）
In robotic-assisted partial nephrectomy, surgeons remove a part of a kidney often due to the presence of a mass. A drop-in ultrasound probe paired to a surgical robot is deployed to execute multiple swipes over the kidney surface to localise the mass and define the margins of resection. This sub-task is challenging and must be performed by a highly skilled surgeon. Automating this sub-task may reduce cognitive load for the surgeon and improve patient outcomes. The overall goal of this work is to autonomously move the ultrasound probe on the surface of the kidney taking advantage of the use of …
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London（伦敦大学学院威康/EPSRC介入与外科科学中心）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）
Semantic tool segmentation in surgical videos is important for surgical scene understanding and computer-assisted interventions as well as for the development of robotic automation. The problem is challenging because different illumination conditions, bleeding, smoke and occlusions can reduce algorithm robustness. At present labelled data for training deep learning models is still lacking for semantic surgical instrument segmentation and in this paper we show that it may be possible to use robot kinematic data coupled with laparoscopic images to alleviate the labelling problem. We propose a ne…
University of Technology Sydney（悉尼科技大学）；Southern University of Science and Technology（南方科技大学）；Shenzhen People’s Hospital（深圳市人民医院）
Total knee arthroplasty (TKA) is a common orthopaedic surgery to replace a damaged knee joint with artificial implants. The inaccuracy of achieving the planned implant position can result in the risk of implant component aseptic loosening, wear out, and even a joint revision, and those failures most of the time occur on the tibial side in the conventional jig-based TKA (CON-TKA). This study aims to precisely evaluate the accuracy of the proximal tibial resection plane intra-operatively in real-time such that the evaluation processing changes very little on the CON-TKA operative procedure. Two …
LapSurgie: Humanoid Robots Performing Surgery via Teleoperated Handheld Laparoscopy
Department of Electrical and Computer Engineering, University of California San Diego（加州大学圣地亚哥分校电气与计算机工程系）；School of Electrical and Computer Engineering, Cornell University（康奈尔大学电气与计算机工程学院）；UC San Diego Health（加州大学圣地亚哥医学中心）
Learning-based Force Sensing and Impedance Matching for Safe Haptic Feedback in Robot-assisted Laparoscopic Surgery
organization= Surgical Robotics Laboratory (SRL), Mechanical Engineering Department, Concordia University , city= Montreal , state= QC , country= Canada；Robotics (SuPER) Centre, Department of Surgery, McGill University , city= Montreal , state= QC , country= Canada；Joyce Posluns Centre for Image Guided Innovation \& Therapeutic Intervention (PCIGITI), The Hospital for Sick Children (SickKids) , city= Toronto , state= ON , country= Canada
Workspace Analysis for Laparoscopic Rectal Surgery : A Preliminary Study
Centre Hospitalier Universitaire（大学中心医院）；CSIR-Central Scientific Instruments Organisation（CSIR中央科学仪器组织）；Academy of Scientific and Innovative Research (AcSIR)（科学与创新研究院）；École Centrale Nantes（南特中央理工学院）；Nantes Université（南特大学）；CNRS（法国国家科学研究中心）；Research Center for Industrial Robots Simulation and Testing（工业机器人仿真与测试研究中心）；Technical University of Cluj-Napoca（克卢日-纳波卡技术大学）
An Open-Source Robotics Research Platform for Autonomous Laparoscopic Surgery
Department of Translational Surgical Oncology, NCT/UCC Dresden, a partnership between DKFZ, MF UKD, TUD, HZDR Dresden, Germany（转化外科肿瘤学系，NCT/UCC德累斯顿，DKFZ、MF UKD、TUD、HZDR德累斯顿的联合体）；Cluster of Excellence-CeTI, TUD, Germany（卓越中心- CeTI，TUD，德国）；Department of Visceral, Thoracic and Vascular Surgery, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD, Germany（visceral、胸腔和血管外科系，医学学院和卡尔·格奥尔格·卡尔医院，TUD，德国）
Point Cloud Segmentation for Autonomous Clip Positioning in Laparoscopic Cholecystectomy on a Phantom
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；HIDSS4Health - Helmholtz Information and Data Science School for Health（亥姆霍兹信息与数据科学健康学校）；Friedrich-Alexander-University, Erlangen-Nuremberg（弗里德里希-亚历山大大学埃尔朗根-纽伦堡）；University Hospital Carl Gustav Carus and Centre for Tactile Internet with Human-in-the-loop (CeTI), Dresden University of Technology（卡尔·古斯塔夫·卡鲁斯大学医院及德累斯顿工业大学触觉互联网人机共融卓越中心）
A novel Integrated Motion Tracking Device (IMTD) for Objective Laparoscopic Training Assessment: Development and Validation
Pprime Institute CNRS（法国国家科学研究中心Pprime研究所）；ENSMA（国立高等机械与航空技术学院）；University of Poitiers（普瓦捷大学）；Mechanical Laboratory of Sousse (LMS)（苏塞机械实验室）；National Engineering School of Sousse（苏塞国立工程师学院）；University of Sousse（苏塞大学）；National Engineering School of Monastir（莫纳斯提尔国立工程师学院）；University of Monastir（莫纳斯提尔大学）；Computer Engineering, Production and Maintenance Laboratory (LGIPM)（计算机工程、生产与维护实验室）；University of Lorraine（洛林大学）
Contact-Stable Deformable Tissue Simulation Using Implicit Integration and Live-Pose Grasp Constraints for Laparoscopic Surgery Robot Policy Evaluation
Seoul National University Hospital（首尔大学医院）；Rosota Inc.（Rosota公司）；Seoul National University College of Medicine（首尔大学医学院）；Seoul National University（首尔大学）；Institute of Convergence Medicine with Innovative Technology, Seoul National University Hospital（首尔大学医院创新技术融合医学研究所）；Eulji University College of Medicine（乙支大学医学院）；Chungang University（中央大学）
MiniTac: An Ultra-Compact 8 mm Vision-Based Tactile Sensor for Enhanced Palpation in Robot-Assisted Minimally Invasive Surgery
State Key Laboratory of General Artificial Intelligence（国家关键人工智能实验室）；Beijing Institute for General Artificial Intelligence（北京通用人工智能研究院）；Institute for Artificial Intelligence（人工智能学院）；Shenyang Institute of Automation（沈阳自动化研究所）；Chinese Academy of Sciences（中国科学院）；PLA General Hospital（中国人民解放军总医院）
Stanford University（斯坦福大学）；Palo Alto Veterans Affairs Health Care System（帕洛阿尔托退伍军人事务医疗系统）；Intuitive Surgical, Inc.（直觉外科公司）；Ben-Gurion University of the Negev（内盖夫本-古里安大学）
The lack of haptic feedback in Robot-assisted Minimally Invasive Surgery (RMIS) is a potential barrier to safe tissue handling during surgery. Bayesian modeling theory suggests that surgeons with experience in open or laparoscopic surgery can develop priors of tissue stiffness that translate to better force estimation abilities during RMIS compared to surgeons with no experience. To test if prior haptic experience leads to improved force estimation ability in teleoperation, 33 participants were assigned to one of three training conditions: manual manipulation, teleoperation with force feedback…
Bridging the Ex-Vivo to In-Vivo Gap: Synthetic Priors for Monocular Depth Estimation in Specular Surgical Environments
RoCAL, Rochester Institute of Technology（罗切斯特理工学院RoCAL实验室）；Department of Otolaryngology Head and Neck Surgery, University of Rochester Medical Center（罗切斯特大学医学中心耳鼻喉科与头颈外科部门）；University of Washington, Department of Otolaryngology–Head and Neck Surgery（华盛顿大学耳鼻喉科与头颈外科部门）
MVP-Tac: A Miniaturized Dual-Modal Vision and Photoelastic Tactile Sensor for Robot-Assisted Minimally Invasive Surgery
Elmore Family School of Electrical and Computer Engineering, Purdue University（普渡大学埃尔莫尔电气与计算机工程学院）；Edwardson School of Industrial Engineering, Purdue University（普渡大学爱德华森工业工程学院）；School of Mechanical Engineering, Purdue University（普渡大学机械工程学院）；Robotics Engineering Technology, Purdue University（普渡大学机器人工程技术学院）；Department of Civil Engineering, Stony Brook University（石溪大学土木工程系）；Department of Mechanical Engineering, Michigan State University（密歇根州立大学机械工程系）
