Autonomous Navigation of an Ultrasound Probe Towards Standard Scan Planes with Deep Reinforcement Learning
The Chinese University of Hong Kong（香港中文大学）；Shenzhen University（深圳大学）；Sonoscape Medical Corp.（开立医疗）；Peking University Shenzhen Hospital（北京大学深圳医院）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of the Chinese University of Hong Kong（香港中文大学深圳研究院）
Keyu Li, Jian Wang, Yangxin Xu, Hao Qin, Dongsheng Liu, Li Liu, Max Q. -H. Meng
备注：
Accepted at ICRA 2021
发表信息：
2021 IEEE International Conference on Robotics and Automation (ICRA), 2021, pp. 8302-8308
Autonomous ultrasound (US) acquisition is an important yet challenging task, as it involves interpretation of the highly complex and variable images and their spatial relationships. In this work, we propose a deep reinforcement learning framework to autonomously control the 6-D pose of a virtual US probe based on real-time image feedback to navigate towards the standard scan planes under the restrictions in real-world US scans. Furthermore, we propose a confidence-based approach to encode the optimization of image quality in the learning process. We validate our method in a simulation environment built with real-world data collected in the US imaging of the spine. Experimental results demonstrate that our method can perform reproducible US probe navigation towards the standard scan plane with an accuracy of $4.91mm/4.65^\circ$ in the intra-patient setting, and accomplish the task in the intra- and inter-patient settings with a success rate of $92\%$ and $46\%$, respectively. The results also show that the introduction of image quality optimization in our method can effectively improve the navigation performance.
Technische Universität München（慕尼黑工业大学）；Sharif University of Technology（谢里夫理工大学）
In this paper we introduce the first reinforcement learning (RL) based robotic navigation method which utilizes ultrasound (US) images as an input. Our approach combines state-of-the-art RL techniques, specifically deep Q-networks (DQN) with memory buffers and a binary classifier for deciding when to terminate the task. Our method is trained and evaluated on an in-house collected data-set of 34 volunteers and when compared to pure RL and supervised learning (SL) techniques, it performs substantially better, which highlights the suitability of RL navigation for US-guided procedures. When testin…
Image-Guided Navigation of a Robotic Ultrasound Probe for Autonomous Spinal Sonography Using a Shadow-aware Dual-Agent Framework
The Chinese University of Hong Kong（香港中文大学）；Shenzhen University（深圳大学）；Southern University of Science and Technology（南方科技大学）；Shenzhen Research Institute of the Chinese University of Hong Kong（香港中文大学深圳研究院）
Goal-conditioned reinforcement learning for ultrasound navigation guidance
King’s College London（伦敦国王学院）；Siemens Healthcare Limited（西门子医疗有限公司）；Siemens Healthineers（西门子医疗）；Siemens Healthineers AG（西门子医疗股份公司）；Carol Davila University of Medicine and Pharmacy Bucharest（布加勒斯特卡罗尔·达维拉医药大学）；Fortis Institute of Medical Sciences（富蒂斯医学院）；Guy’s and St Thomas’ NHS Foundation Trust（盖伊和圣托马斯国民保健信托基金）
Division of Science and Engineering, Penn State Abington（宾州州立大学阿宾顿分校科学与工程学院）
This paper investigates different methods to detect obstacles ahead of a robot using a camera in the robot, an aerial camera, and an ultrasound sensor. We also explored various efficient path finding methods for the robot to navigate to the target source. Single and multi-iteration angle-based navigation algorithms were developed. The theta-based path finding algorithms were compared with the Dijkstra Algorithm and their performance were analyzed.
End-to-End Framework Integrating Generative AI and Deep Reinforcement Learning for Autonomous Ultrasound Scanning
organization= Concordia Institute for Information Systems Engineering, Concordia University , city= Montreal , country= Canada；organization= Department of Mechanical, Industrial；Aerospace Engineering, Concordia University , city= Montreal , country= Canada；organization= Department of Medicine, Laval University , city= Quebec , country= Canada；organization= Department of Electrical, Computer；Software Engineering, Ontario Tech University , city= Oshawa , country= Canada；organization= Department of Computer Science, Khalifa University , city= Abu Dhabi , country= UAE；organization= Artificial Intelligence \& Cyber Systems Research Center, Department of CSM, Lebanese American University , city= Beirut , country= Lebanon；organization= Department of Electrical Engineering；Computer Science, Khalifa University , city= Abu Dhabi , country= UAE
Enhancing Surgical Robots with Embodied Intelligence for Autonomous Ultrasound Scanning
Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Microrobots are considered as promising tools for biomedical applications. However, the imaging of them becomes challenges in order to be further applied on in vivo environments. Here we report the magnetic navigation of a paramagnetic nanoparticle based swarm using ultrasound images. The swarm can be generated using simple rotating magnetic fields, resulting in a region containing particles with a high area density. Ultrasound images of the swarm shows a periodic changing of imaging contrast. The reason for such dynamic contrast has been analyzed and experimental results are presented. Moreov…
Tsinghua University（清华大学）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
Ultrasound scanning robots enable the automatic imaging of a patient's internal organs by maintaining close contact between the ultrasound probe and the patient's body during a scanning procedure. Comprehensive, high-quality ultrasound scans are essential for providing the patient with an accurate diagnosis and effective treatment plan. An ultrasound scanning robot usually works in a doctor-robot co-existing environment, hence both efficiency and safety during the collaboration should be considered. In this paper, we propose a novel multi-modal control scheme for ultrasound scanning robots, in…
The Hamlyn Centre for Robotic Surgery, Imperial College London, UK（帝国理工学院哈姆林机器人外科中心）；Department of Surgery and Cancer, Imperial College London, UK（帝国理工学院外科与癌症系）
Intraoperative ultrasound facilitates localisation of tumour boundaries during minimally invasive procedures. Autonomous ultrasound scanning systems have been recently proposed to improve scanning accuracy and reduce surgeons' cognitive load. However, current methods mainly consider static scanning environments typically with the probe pressing against the tissue surface. In this work, a motion-compensated autonomous ultrasound scanning system using the da Vinci Research Kit (dVRK) is proposed. An optimal scanning trajectory is generated considering both the tissue surface shape and the ultras…
ForceU-VLA: A Force-Aware Vision-Language-Action Model for Embodied Ultrasound Scanning
Faculty of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Information Science and Engineering, Shandong University（山东大学信息科学与工程学院）；Innovation School of Artificial Intelligence, Hefei University of Technology（合肥工业大学人工智能创新学院）
UltraDP: Generalizable Carotid Ultrasound Scanning with Force-Aware Diffusion Policy
Department of Automation, Tsinghua University（自动化系，清华大学）；Department of Surgery, Chow Yuk Ho Technology Centre for Innovative Medicine, Li Ka Shing Institute of Health Science and Multi-scale Medical Robotics Center, The Chinese University of Hong Kong, Hong Kong（外科系，创新医学技术中心，利文斯科健康科学研究所及多尺度医学机器人中心，香港中文大学，香港）
The University of Sydney（悉尼大学）；University of California San Diego（加州大学圣地亚哥分校）
Ultrasound is progressing toward becoming an affordable and versatile solution to medical imaging. With the advent of COVID-19 global pandemic, there is a need to fully automate ultrasound imaging as it requires trained operators in close proximity to patients for a long period of time, therefore increasing risk of infection. In this work, we investigate the important yet seldom-studied problem of scan target localization, under the setting of lung ultrasound imaging. We propose a purely vision-based, data driven method that incorporates learning-based computer vision techniques. We combine a …
US-VLA: An Ultrasound Vision-Language-Action Model for Embodied Abdomina
Faculty of Computer Science and Technology, Ocean University of China（中国海洋大学计算机科学与技术学院）；School of Information Science and Engineering, Shandong University（山东大学信息科学与工程学院）；the Qilu Second Hospital of Shandong University（山东大学第二齐鲁医院）；Innovation School of Artificial Intelligence, Hefei University of Technology（合肥工业大学人工智能创新学院）
Shape Completion and Real-Time Visualization in Robotic Ultrasound Spine Acquisitions
Computer Aided Medical Procedures, Technical University of Munich（技术大学慕尼黑计算机辅助医疗程序）；Konrad Zuse School of Excellence in Reliable AI (relAI)（Konrad Zuse卓越人工智能学校）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Task-Relevant Feature-Dynamics Fidelity Enables Zero-Shot Sim-to-Real Transfer for Robotic Ultrasound Scanning
The Chinese University of Hong Kong（香港中文大学）；Great Bay University（大湾区大学）；Dongguan Great Bay Institute for Advanced Study（东莞大湾区先进研究院）；The University of Hong Kong（香港大学）；Southern University of Science and Technology（南方科技大学）
Technical University of Munich（慕尼黑工业大学）；Zhejiang University（浙江大学）；German Heart Center Munich（慕尼黑德国心脏中心）
Robotic ultrasound (US) imaging has been seen as a promising solution to overcome the limitations of free-hand US examinations, i.e., inter-operator variability. However, the fact that robotic US systems cannot react to subject movements during scans limits their clinical acceptance. Regarding human sonographers, they often react to patient movements by repositioning the probe or even restarting the acquisition, in particular for the scans of anatomies with long structures like limb arteries. To realize this characteristic, we proposed a vision-based system to monitor the subject's movement an…
University Hospital Schleswig-Holstein（石勒苏益格-荷尔斯泰因大学医院）；University of Lübeck（吕贝克大学）
The first choice in diagnostic imaging for patients suffering from peripheral arterial disease is 2D ultrasound (US). However, for a proper imaging process, a skilled and experienced sonographer is required. Additionally, it is a highly user-dependent operation. A robotized US system that autonomously scans the peripheral arteries has the potential to overcome these limitations. In this work, we extend a previously proposed system by a hierarchical image analysis pipeline based on convolutional neural networks in order to control the robot. The system was evaluated by checking its feasibility …
Technische Universität München（慕尼黑工业大学）；KUKA Roboter GmbH（库卡机器人有限公司）；Johns Hopkins University（约翰斯·霍普金斯大学）
Robotic ultrasound has the potential to assist and guide physicians during interventions. In this work, we present a set of methods and a workflow to enable autonomous MRI-guided ultrasound acquisitions. Our approach uses a structured-light 3D scanner for patient-to-robot and image-to-patient calibration, which in turn is used to plan 3D ultrasound trajectories. These MRI-based trajectories are followed autonomously by the robot and are further refined online using automatic MRI/US registration. Despite the low spatial resolution of structured light scanners, the initial planned acquisition pa…
Visuotactile and Explicitly Force-Controlled Robotic Ultrasound for Abdominal Volumetric Reconstruction
Stanford Robotics Laboratory, Computer Science Department, Stanford University（斯坦福机器人实验室、计算机科学系、斯坦福大学）；Department of Radiology, School of Medicine, Stanford University（放射科、医学院、斯坦福大学）
Action-Conditioned World Model for Goal Plane Guidance in Robotic Neck Ultrasound
Department of Mechanical and Automation Engineering, Chinese University of Hong Kong（香港中文大学机械与自动化工程学系）；Department of Biomedical Engineering, City University of Hong Kong（香港城市大学生物医学工程学系）；Centre for Artificial Intelligence and Robotics Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences（中国科学院香港创新研究院人工智能与机器人中心）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Department of Surgical and Interventional Engineering, King’s College London（伦敦国王大学外科与介入工程学系）；Department of Vascular Ultrasound, Xuanwu Hospital, the Capital Medical University（首都医科大学宣武医院血管超声科）
Autonomous Robotic Ultrasound System for Liver Follow-up Diagnosis: Pilot Phantom Study
School of Engineering and Applied Sciences, Harvard University（哈佛大学工程与应用科学学院）；Center for Advanced Medical Computing and Analysis (CAMCA), Massachusetts General Hospital（麻省总医院高级医学计算与分析中心）；Harvard Medical School（哈佛医学院）
Technical University of Munich（慕尼黑工业大学）；Shanghai Jiao Tong University（上海交通大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Robotic ultrasound (US) imaging aims at overcoming some of the limitations of free-hand US examinations, e.g. difficulty in guaranteeing intra- and inter-operator repeatability. However, due to anatomical and physiological variations between patients and relative movement of anatomical substructures, it is challenging to robustly generate optimal trajectories to examine the anatomies of interest, in particular, when they comprise articulated joints. To address this challenge, this paper proposes a vision-based approach allowing autonomous robotic US limb scanning. To this end, an atlas MRI tem…
Noah’s Ark Lab, Huawei Technologies Canada, Ltd.（华为技术加拿大有限公司诺亚方舟实验室）；University of Alberta（阿尔伯塔大学）
We propose a method to tackle the problem of mapless collision-avoidance navigation where humans are present using 2D laser scans. Our proposed method uses ego-safety to measure collision from the robot's perspective while social-safety to measure the impact of our robot's actions on surrounding pedestrians. Specifically, the social-safety part predicts the intrusion impact of our robot's action into the interaction area with surrounding humans. We train the policy using reinforcement learning on a simple simulator and directly evaluate the learned policy in Gazebo and real robot tests. Experi…
Saint Petersburg State University（圣彼得堡国立大学）；University of New South Wales（新南威尔士大学）
A single nonholonomic under-actuated mobile robot with a bounded control range travels in a 3D workspace. This workspace also hosts a compact unknown domain with a smooth boundary. The robot has access to the horizontal distance to this domain and to a certain direction (typically, vertical) in the space, along with its coordinate (typically, altitude) in this direction. We present a new navigation law that drives the robot to the desired distance to the domain and ensures subsequent full scan of its surface within a desired range of "altitudes". The proposed strategy is computationally inexpe…
AIA-UltraNeRF:Acoustic-Impedance-Aware Neural Radiance Field with Hash Encodings for Robotic Ultrasound Reconstruction and Localization
School of Management, Hefei University of Technology（合肥工业大学管理学院）；First Affiliated Hospital of University of Science and Technology of China（中国科学技术大学第一附属医院）；Department of Ultrasound Diagnosis, the Second Xiangya Hospital of Central South University（中南大学湘雅医学院第二医院超声诊断科）；Intelligent Ultrasound Division of VINNO Technology Co., Ltd.（VINNO技术有限公司智能超声事业部）；Department of Automation, Tsinghua University（清华大学自动化系）
SAMe: A Semantic Anatomy Mapping Engine for Robotic Ultrasound
School of Computer Science, Wuhan University（武汉大学计算机学院）；Hubei Center for Applied Mathematics, Wuhan University（湖北应用数学中心，武汉大学）；Department of Ultrasound, The Central Hospital of Wuhan（武汉市中心医院超声科）；Department of Medical Ultrasound, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology（同济医院，同济医学院，华中科技大学医学影像科）；Department of Ultrasound in Medicine, Renmin Hospital of Wuhan University（武汉大学仁医医院医学超声科）；University Hospital, Johann-Wolfgang-Goethe University Frankfurt am Main（法兰克福歌德大学医学院大学医院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Contact-Adaptive Robotic Ultrasound Probe Control for Tissue Exploration and Continuous Task-Relevant Visualization Using Robot-Free Image-Motion Demonstration
Seoul National University Hospital（首尔大学医院）；Rosota Inc.（Rosota公司）；Seoul National University College of Medicine（首尔大学医学院）；Seoul National University（首尔大学）；Eulji University College of Medicine（乙支大学医学院）
Robot-Assisted Deep Venous Thrombosis Ultrasound Examination using Virtual Fixture
Chair for Computer Aided Medical Procedures and Augmented Reality (CAMP), Technical University of Munich (TUM)（慕尼黑工业大学计算机辅助医疗程序与增强现实主席（CAMP））；Bristol Robotics Laboratory, University of the West of England（西英格兰大学布里斯托尔机器人实验室）；College of Biosystems of Engineering and Food Science, Zhejiang University（浙江大学生物系统工程与食品科学学院）；Institute of Biological and Medical Imaging, Helmholtz Zentrum Munchen（慕尼黑亥姆霍兹中心生物医学影像研究所）；Department for Vascular and Endovascular Surgery, rechts der Isar University Hospital, Technical University of Munich（慕尼黑工业大学伊斯河畔大学医院血管与血管外科）
In Minimally Invasive Surgery (MIS), tissue scanning with imaging probes is required for subsurface visualisation to characterise the state of the tissue. However, scanning of large tissue surfaces in the presence of deformation is a challenging task for the surgeon. Recently, robot-assisted local tissue scanning has been investigated for motion stabilisation of imaging probes to facilitate the capturing of good quality images and reduce the surgeon's cognitive load. Nonetheless, these approaches require the tissue surface to be static or deform with periodic motion. To eliminate these assumpt…
