Task automation of surgical robot has the potentials to improve surgical efficiency. Recent reinforcement learning (RL) based approaches provide scalable solutions to surgical automation, but typically require extensive data collection to solve a task if no prior knowledge is given. This issue is known as the exploration challenge, which can be alleviated by providing expert demonstrations to an RL agent. Yet, how to make effective use of demonstration data to improve exploration efficiency still remains an open challenge. In this work, we introduce Demonstration-guided EXploration (DEX), an efficient reinforcement learning algorithm that aims to overcome the exploration problem with expert demonstrations for surgical automation. To effectively exploit demonstrations, our method estimates expert-like behaviors with higher values to facilitate productive interactions, and adopts non-parametric regression to enable such guidance at states unobserved in demonstration data. Extensive experiments on $10$ surgical manipulation tasks from SurRoL, a comprehensive surgical simulation platform, demonstrate significant improvements in the exploration efficiency and task success rates of our method. Moreover, we also deploy the learned policies to the da Vinci Research Kit (dVRK) platform to show the effectiveness on the real robot. Code is available at https://github.com/med-air/DEX.
Robotic surgery has increased the domain of surgeries possible. Several examples of partial surgical automation have been seen in the past decade. We break down the path of automation tasks into features required and provide a checklist that can help reach higher levels of surgical automation. Finally, we discuss the current challenges and advances required to make this happen.
DARIL: When Imitation Learning outperforms Reinforcement Learning in Surgical Action Planning
Surgical and Interventional Engineering, King's College London, London, UK（外科与介入工程，伦敦国王学院，伦敦，英国）；Interventional Engineering, King's College London, London, UK（介入工程，伦敦国王学院，伦敦，英国）
该研究在 CholecT50 上比较手术动作规划中的 IL 与 RL，提出 DARIL 基线，发现其显著优于三种 RL 方法，并揭示专家测试分布对 IL 的系统性偏好。
Tendon-driven robots, a type of continuum robot, have the potential to reduce the invasiveness of surgery by enabling access to difficult-to-reach anatomical targets. In the future, the automation of surgical tasks for these robots may help reduce surgeon strain in the face of a rapidly growing population. However, directly encoding surgical tasks and their associated context for these robots is infeasible. In this work we take steps toward a system that is able to learn to successfully perform context-dependent surgical tasks by learning directly from a set of expert demonstrations. We presen…
NOVA LINCS（NOVA LINCS实验室）；Universidade da Beira Interior（贝拉 Interior 大学）
With the fast improvement of machine learning, reinforcement learning (RL) has been used to automate human tasks in different areas. However, training such agents is difficult and restricted to expert users. Moreover, it is mostly limited to simulation environments due to the high cost and safety concerns of interactions in the real world. Demonstration Learning is a paradigm in which an agent learns to perform a task by imitating the behavior of an expert shown in demonstrations. It is a relatively recent area in machine learning, but it is gaining significant traction due to having tremendou…
Surgical Task Automation Using Actor-Critic Frameworks and Self-Supervised Imitation Learning
University of Edinburgh（爱丁堡大学）；Institute for Imaging, Data and Communications（成像、数据与通信研究所）；School of Engineering（工程学院）；TECNALIA（泰克纳利亚）；University of Strathclyde（思克莱德大学）；Centre for Precision Manufacturing（精密制造中心）；Department of Biomedical Engineering（生物医学工程系）
Johns Hopkins University（约翰斯·霍普金斯大学）；Wilmer Eye Institute at the Johns Hopkins Hospital（约翰斯·霍普金斯医院威尔默眼科研究所）
A fundamental challenge in retinal surgery is safely navigating a surgical tool to a desired goal position on the retinal surface while avoiding damage to surrounding tissues, a procedure that typically requires tens-of-microns accuracy. In practice, the surgeon relies on depth-estimation skills to localize the tool-tip with respect to the retina in order to perform the tool-navigation task, which can be prone to human error. To alleviate such uncertainty, prior work has introduced ways to assist the surgeon by estimating the tool-tip distance to the retina and providing haptic or auditory fee…
City University of Hong Kong（香港城市大学）；The Chinese University of Hong Kong（香港中文大学）
Automatic laparoscope motion control is fundamentally important for surgeons to efficiently perform operations. However, its traditional control methods based on tool tracking without considering information hidden in surgical scenes are not intelligent enough, while the latest supervised imitation learning (IL)-based methods require expensive sensor data and suffer from distribution mismatch issues caused by limited demonstrations. In this paper, we propose a novel Imitation Learning framework for Laparoscope Control (ILLC) with reinforcement learning (RL), which can efficiently learn the con…
SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy
Department of Computer Science and Engineering, The Chinese University of Hong Kong（香港中文大学计算机科学与工程系）；Department of Thoracic Surgery, Peking University People’s Hospital（北京大学人民医院胸外科）；Thoracic Oncology Institute, Peking University People’s Hospital（北京大学人民医院胸部肿瘤研究所）；Research Unit of Intelligence Diagnosis and Treatment in Early Non-small Cell Lung Cancer, Chinese Academy of Medical Sciences（中国医学科学院早期非小细胞肺癌智能诊断与治疗研究组）；Institute of Advanced Clinical Medicine, Peking University（北京大学先进临床医学院）；Beijing Key Laboratory of Innovative Application of Big Data in Lung Cancer, Peking University People’s Hospital（北京大学人民医院肺癌大数据创新应用北京市重点实验室）
Robotic Surgical Assistants (RSAs) are commonly used to perform minimally invasive surgeries by expert surgeons. However, long procedures filled with tedious and repetitive tasks such as suturing can lead to surgeon fatigue, motivating the automation of suturing. As visual tracking of a thin reflective needle is extremely challenging, prior work has modified the needle with nonreflective contrasting paint. As a step towards automation of a suturing subtask without modifying the needle, we propose HOUSTON: Handoff of Unmodified, Surgical, Tool-Obstructed Needles, a problem and algorithm that us…
The control and task automation of robotic surgical system is very challenging, especially in soft tissue manipulation, due to the unpredictable deformations. Thus, an accurate simulator of soft tissues with the ability of interacting with robot manipulators is necessary. In this work, we propose a novel 2D simulation framework for tool-tissue interaction. This framework continuously tracks the motion of manipulator and simulates the tissue deformation in presence of collision detection. The deformation energy can be computed for the control and planning task.
Overcoming Imperfect Kinematics in Surgical Robotics Through Sim-to-Real Visuomotor Learning
Hamlyn Centre for Robotic Surgery, Institute of Global Health Innovation, Imperial College London（伦敦帝国理工学院全球健康创新研究所哈姆林机器人手术中心）；Department of Surgery and Cancer, Imperial College London（伦敦帝国理工学院外科与癌症系）；Department of Mechanical Engineering, Imperial College London（伦敦帝国理工学院机械工程系）
Robotic System for Chemical Experiment Automation with Dual Demonstration of End-effector and Jig Operations
Division of Information, Science Graduate School of Science and Technology（信息科学研究生院）；Nara Institute of Science and Technology（奈良科学技术大学）；Materials Informatics Initiative, RD technology and digital transformation center（材料信息学计划，RD技术与数字化转型中心）；JSR Corporation（JSR公司）；Fine Chemical Process Dept., JSR Corporation（JSR公司精细化学工艺部门）；Department of Chemistry Faculty of Science and Technology（科学与技术学部化学系）；Keio University（庆应义塾大学）；Data Science Center（数据科学中心）；Division of Materials Science, Graduate School of Science and Technology（材料科学系，研究生院）
Vitreoretinal surgery is challenging even for expert surgeons owing to the delicate target tissues and the diminutive workspace in the retina. In addition to improved dexterity and accuracy, robot assistance allows for (partial) task automation. In this work, we propose a strategy to automate the motion of the light guide with respect to the surgical instrument. This automation allows the instrument's shadow to always be inside the microscopic view, which is an important cue for the accurate positioning of the instrument in the retina. We show simulations and experiments demonstrating that the…
Supervised Mixture-of-Experts for Surgical Grasping and Retraction
Department of Translational Surgical Oncology, NCT/UCC Dresden, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD, Dresden, Germany（translational Surgical Oncology部门，NCT/UCC Dresden，医学院和Carl Gustav Carus大学医院，TUD，德累斯顿，德国）；National Center for Tumor Diseases (NCT), NCT/UCC Dresden, a partnership between DKFZ, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD, and HZDR, Dresden, Germany（肿瘤疾病国家中心（NCT），NCT/UCC德累斯顿，DKFZ、医学院和Carl Gustav Carus大学医院、TUD以及HZDR之间的合作，德累斯顿，德国）；Faculty of Computer Science, TUD, Dresden, Germany（计算机科学系，TUD，德累斯顿，德国）；The Center for Tactile Internet with Human-in-the-Loop (CeTI), TUD, Dresden, Germany（带有Human-in-the-Loop的触觉互联网中心（CeTI），TUD，德累斯顿，德国）；Department of Visceral, Thoracic and Vascular Surgery, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD, Dresden, Germany（visceral、胸腔和血管外科部门，医学院和Carl Gustav Carus大学医院，TUD，德累斯顿，德国）
Learning-based Hierarchical Tracheal Anatomy Understanding from Sparse Surgical Demonstration Annotations for Ultrasound Robots
The Chinese University of Hong Kong（香港中文大学）；Imperial College London（伦敦帝国学院）；Hong Kong Institute of Science and Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院）；SLAI Shenzhen Loop Area Institute（深圳环域先进人工智能研究院）
The Chinese University of Hong Kong（香港中文大学）；The Johns Hopkins University（约翰斯·霍普金斯大学）
Robot-assisted surgery has become progressively more and more popular due to its clinical advantages. In the meanwhile, the artificial intelligence and augmented reality in robotic surgery are developing rapidly and receive lots of attention. However, current methods have not discussed the coherent integration of AI and AR in robotic surgery. In this paper, we develop a novel system by seamlessly merging artificial intelligence module and augmented reality visualization to automatically generate the surgical guidance for robotic surgery education. Specifically, we first leverage reinforcement …
Wellcome/EPSRC Centre for Interventional and Surgical Sciences (WEISS), University College London（伦敦大学学院威康/EPSRC介入与外科科学中心）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）
Semantic tool segmentation in surgical videos is important for surgical scene understanding and computer-assisted interventions as well as for the development of robotic automation. The problem is challenging because different illumination conditions, bleeding, smoke and occlusions can reduce algorithm robustness. At present labelled data for training deep learning models is still lacking for semantic surgical instrument segmentation and in this paper we show that it may be possible to use robot kinematic data coupled with laparoscopic images to alleviate the labelling problem. We propose a ne…
Learning Autonomous Surgical Irrigation and Suction with the da Vinci Research Kit Using Reinforcement Learning
Department of Electrical and Computer Engineering, University of Alberta（电气与计算机工程系，阿尔伯塔大学）；Department of Biomedical Engineering, University of Alberta（生物医学工程系，阿尔伯塔大学）
Diffusion Stabilizer Policy for Automated Surgical Robot Manipulations
UM-SJTU Joint Intuitute（UM-SJTU联合学院）；Shanghai Jiao Tong University（上海交通大学）；Department of Computer Science and Engineering（计算机科学与工程系）；The Chinese University of Hong Kong（香港中文大学）；Department of Automation（自动化系）
UC Berkeley（加州大学伯克利分校）；UCSF East Bay（加州大学旧金山分校东湾）；SRI International（斯坦福国际研究院）
Automation of surgical subtasks using cable-driven robotic surgical assistants (RSAs) such as Intuitive Surgical's da Vinci Research Kit (dVRK) is challenging due to imprecision in control from cable-related effects such as cable stretching and hysteresis. We propose a novel approach to efficiently calibrate such robots by placing a 3D printed fiducial coordinate frames on the arm and end-effector that is tracked using RGBD sensing. To measure the coupling and history-dependent effects between joints, we analyze data from sampled trajectories and consider 13 approaches to modeling. These model…
Despite significant developments in the design of surgical robots and automated techniques for objective evaluation of surgical skills, there are still challenges in ensuring safety in robot-assisted minimally-invasive surgery (RMIS). This paper presents a runtime monitoring system for the detection of executional errors during surgical tasks through the analysis of kinematic data. The proposed system incorporates dual Siamese neural networks and knowledge of surgical context, including surgical tasks and gestures, their distributional similarities, and common error modes, to learn the differe…
Cosmos-H-Surgical: Learning Surgical Robot Policies from Videos via World Modeling
NVIDIA；The Chinese University of Hong Kong（香港中文大学）；Sung Kyun Kwan University（全州大学）；Wenzhou Medical University（温州医学院）；National University of Singapore（新加坡国立大学）；Ruijin Hospital（瑞金医院）
