Toward Autonomous Soft Robotic Endovascular Navigation via Imitation Learning
Johns Hopkins University（约翰霍普金斯大学）；Stanford University（斯坦福大学）；McGill University（麦吉尔大学）；University of Maryland（马里兰大学）；Swiss Federal Institute of Technology in Lausanne (EPFL)（日内瓦联邦理工学院（EPFL））；ETH Zurich（苏黎世联邦理工学院）
Noah Barnes, Ji Woong Kim, Lingyun Di, Hannah Qu, Anuruddha Bhattacharjee, Miroslaw Janowski, Dheeraj Gandhi, Bailey Felix, Shaopeng Jiang, Olivia Young, Mark F… 展开作者
Noah Barnes, Ji Woong Kim, Lingyun Di, Hannah Qu, Anuruddha Bhattacharjee, Miroslaw Janowski, Dheeraj Gandhi, Bailey Felix, Shaopeng Jiang, Olivia Young, Mark Fuge, Ryan D. Sochol, Jeremy D. Brown, Axel Krieger
In endovascular surgery, endovascular interventionists push a thin tube called a catheter, guided by a thin wire to a treatment site inside the patient's blood vessels to treat various conditions such as blood clots, aneurysms, and malformations. Robotic guidewires can enhance maneuverability but are difficult to model and control. Autonomous soft robotic guidewire navigation has the potential to overcome these challenges, increasing the precision and safety of endovascular navigation. As a first step, we establish a large-scale, 2D-projected environment for autonomous navigation. In other surgical domains, end-to-end imitation learning has shown promising results. Thus, we develop a transformer-based imitation learning framework with goal conditioning, relative action outputs, and automatic contrast dye injections to enable generalizable soft robot navigation in an aneurysm targeting task. We train the policy on 36 different modular bifurcated geometries, generating 647 total demonstrations under simulated fluoroscopy, and evaluate it on three previously unseen vascular geometries. The policy reaches the aneurysm with a success rate of 83% on the unseen geometries, outperforming several baselines. In addition, ablation and baseline studies evaluate the effectiveness of each design and data collection choice. Lastly, we extend the policy to achieve 75% success on an unseen patient-derived geometry. Project website: https://softrobotnavigation.github.io/
University of California, Berkeley（加州大学伯克利分校）；Robotics at Google（谷歌机器人学）
Imitation learning is a popular approach for training visual navigation policies. However, collecting expert demonstrations for legged robots is challenging as these robots can be hard to control, move slowly, and cannot operate continuously for a long time. Here, we propose a zero-shot imitation learning approach for training a visual navigation policy on legged robots from human (third-person perspective) demonstrations, enabling high-quality navigation and cost-effective data collection. However, imitation learning from third-person demonstrations raises unique challenges. First, these demo…
We focus on the utilisation of reactive trajectory imitation controllers for goal-directed mobile robot navigation. We propose a topological navigation graph (TNG) - an imitation-learning-based framework for navigating through environments with intersecting trajectories. The TNG framework represents the environment as a directed graph composed of deep neural networks. Each vertex of the graph corresponds to a trajectory and is represented by a trajectory identification classifier and a trajectory imitation controller. For trajectory following, we propose the novel use of neural object detectio…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Army Research Laboratory（陆军研究实验室）；Sony AI（索尼人工智能）
While imitation learning for vision based autonomous mobile robot navigation has recently received a great deal of attention in the research community, existing approaches typically require state action demonstrations that were gathered using the deployment platform. However, what if one cannot easily outfit their platform to record these demonstration signals or worse yet the demonstrator does not have access to the platform at all? Is imitation learning for vision based autonomous navigation even possible in such scenarios? In this work, we hypothesize that the answer is yes and that recent …
In recent years, great efforts have been devoted to deep imitation learning for autonomous driving control, where raw sensory inputs are directly mapped to control actions. However, navigating through densely populated intersections remains a challenging task due to uncertainty caused by uncertain traffic participants. We focus on autonomous navigation at crowded intersections that require interaction with pedestrians. A multi-task conditional imitation learning framework is proposed to adapt both lateral and longitudinal control tasks for safe and efficient interaction. A new benchmark called…
ETH Zurich（苏黎世联邦理工学院）；Max Planck ETH Center for Learning Systems（马克斯·普朗克-苏黎世联邦理工学习系统中心）
This work presents a case study of a learning-based approach for target driven map-less navigation. The underlying navigation model is an end-to-end neural network which is trained using a combination of expert demonstrations, imitation learning (IL) and reinforcement learning (RL). While RL and IL suffer from a large sample complexity and the distribution mismatch problem, respectively, we show that leveraging prior expert demonstrations for pre-training can reduce the training time to reach at least the same level of performance compared to plain RL by a factor of 5. We present a thorough ev…
Korea Advanced Institute of Science and Technology（韩国科学技术院）
This paper presents a vision-based modularized drone racing navigation system that uses a customized convolutional neural network (CNN) for the perception module to produce high-level navigation commands and then leverages a state-of-the-art planner and controller to generate low-level control commands, thus exploiting the advantages of both data-based and model-based approaches. Unlike the state-of-the-art method which only takes the current camera image as the CNN input, we further add the latest three drone states as part of the inputs. Our method outperforms the state-of-the-art method in …
Deep imitation learning enables robots to learn from expert demonstrations to perform tasks such as lane following or obstacle avoidance. However, in the traditional imitation learning framework, one model only learns one task, and thus it lacks of the capability to support a robot to perform various different navigation tasks with one model in indoor environments. This paper proposes a new framework, Shared Multi-headed Imitation Learning(SMIL), that allows a robot to perform multiple tasks with one model without switching among different models. We model each task as a sub-policy and design …
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；National University of Defense Technology（国防科技大学）；University of Maryland, College Park（马里兰大学帕克分校）
We present a target-driven navigation system to improve mapless visual navigation in indoor scenes. Our method takes a multi-view observation of a robot and a target as inputs at each time step to provide a sequence of actions that move the robot to the target without relying on odometry or GPS at runtime. The system is learned by optimizing a combinational objective encompassing three key designs. First, we propose that an agent conceives the next observation before making an action decision. This is achieved by learning a variational generative module from expert demonstrations. We then prop…
The Hong Kong University of Science and Technology（香港科技大学）；Albert Ludwig University of Freiburg（阿尔伯特·路德维希 Freiburg 大学）
We present an approach for mobile robots to learn to navigate in dynamic environments with pedestrians via raw depth inputs, in a socially compliant manner. To achieve this, we adopt a generative adversarial imitation learning (GAIL) strategy, which improves upon a pre-trained behavior cloning policy. Our approach overcomes the disadvantages of previous methods, as they heavily depend on the full knowledge of the location and velocity information of nearby pedestrians, which not only requires specific sensors, but also the extraction of such state information from raw sensory input could consu…
Stanford University（斯坦福大学）；Everyday Robots（Everyday机器人公司）；Robotics at Google（谷歌机器人部门）；Apple ML Research（苹果机器学习研究院）；Adobe Research（奥多比研究院）
As robots increasingly enter human-centered environments, they must not only be able to navigate safely around humans, but also adhere to complex social norms. Humans often rely on non-verbal communication through gestures and facial expressions when navigating around other people, especially in densely occupied spaces. Consequently, robots also need to be able to interpret gestures as part of solving social navigation tasks. To this end, we present Gesture2Path, a novel social navigation approach that combines image-based imitation learning with model-predictive control. Gestures are interpre…
Artificial Intelligence in the Autonomous Navigation of Endovascular Interventions: A Systematic Review
School of Biomedical Engineering & Imaging Sciences（生物医学工程与影像科学学院）；Kings College London（伦敦国王学院）；AIBE；Friedrich-Alexander University Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）
University of California, Santa Barbara（加州大学圣巴巴拉分校）；Microsoft Research, Redmond（微软研究院雷德蒙德分院）；Duke University（杜克大学）
Vision-language navigation (VLN) is the task of navigating an embodied agent to carry out natural language instructions inside real 3D environments. In this paper, we study how to address three critical challenges for this task: the cross-modal grounding, the ill-posed feedback, and the generalization problems. First, we propose a novel Reinforced Cross-Modal Matching (RCM) approach that enforces cross-modal grounding both locally and globally via reinforcement learning (RL). Particularly, a matching critic is used to provide an intrinsic reward to encourage global matching between instruction…
SGN-CIRL: Scene Graph-based Navigation with Curriculum, Imitation, and Reinforcement Learning
Moscow Institute of Physics and Technology (MIPT)（莫斯科物理技术学院）；Federal Research Center for Computer Science and Control of Russian Academy of Sciences（俄罗斯科学院计算机科学与控制联邦研究中心）；Artificial Intelligence Research Institute (AIRI)（人工智能研究院）
In this work, we present a method for obtaining an implicit objective function for vision-based navigation. The proposed methodology relies on Imitation Learning, Model Predictive Control (MPC), and an interpretation technique used in Deep Neural Networks. We use Imitation Learning as a means to do Inverse Reinforcement Learning in order to create an approximate cost function generator for a visual navigation challenge. The resulting cost function, the costmap, is used in conjunction with MPC for real-time control and outperforms other state-of-the-art costmap generators in novel environments.…
We introduce a method for following high-level navigation instructions by mapping directly from images, instructions and pose estimates to continuous low-level velocity commands for real-time control. The Grounded Semantic Mapping Network (GSMN) is a fully-differentiable neural network architecture that builds an explicit semantic map in the world reference frame by incorporating a pinhole camera projection model within the network. The information stored in the map is learned from experience, while the local-to-world transformation is computed explicitly. We train the model using DAggerFM, a …
In mobile manipulation (MM), robots can both navigate within and interact with their environment and are thus able to complete many more tasks than robots only capable of navigation or manipulation. In this work, we explore how to apply imitation learning (IL) to learn continuous visuo-motor policies for MM tasks. Much prior work has shown that IL can train visuo-motor policies for either manipulation or navigation domains, but few works have applied IL to the MM domain. Doing this is challenging for two reasons: on the data side, current interfaces make collecting high-quality human demonstra…
University of Maryland, College Park（马里兰大学帕克分校）；Microsoft Research, Redmond（微软研究院雷德蒙德分院）
We present Vision-based Navigation with Language-based Assistance (VNLA), a grounded vision-language task where an agent with visual perception is guided via language to find objects in photorealistic indoor environments. The task emulates a real-world scenario in that (a) the requester may not know how to navigate to the target objects and thus makes requests by only specifying high-level end-goals, and (b) the agent is capable of sensing when it is lost and querying an advisor, who is more qualified at the task, to obtain language subgoals to make progress. To model language-based assistance…
Imitation learning enables high-fidelity, vision-based learning of policies within rich, photorealistic environments. However, such techniques often rely on traditional discrete-time neural models and face difficulties in generalizing to domain shifts by failing to account for the causal relationships between the agent and the environment. In this paper, we propose a theoretical and experimental framework for learning causal representations using continuous-time neural networks, specifically over their discrete-time counterparts. We evaluate our method in the context of visual-control learning…
Vision-Language Procedural Reasoning for Context-Aware Reward Modeling of Robotic Endovascular Guidewire Navigation
Department of Control Science and Engineering, College of Electronic and Information Engineering, and Shanghai Institute of Intelligent Science and Technology, Tongji University（同济大学电子与信息工程学院控制科学与工程系，以及上海智能科学与技术研究院）；Department of Electronic Engineering, Faculty of Engineering, The Chinese University of Hong Kong（香港中文大学工程学院电子工程系）；Shanghai Operation Robot Co., Ltd.（上海微创医疗机器人（集团）股份有限公司）
HTNav: A Hybrid Navigation Framework with Tiered Structure for Urban Aerial Vision-and-Language Navigation
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education（教育部脑机智能技术重点实验室）
Intel Labs（英特尔实验室）；Computer Vision Center & Univ. Autonoma de Barcelona（计算机视觉中心与巴塞罗那自治大学）；King Abdullah University of Science & Technology（阿卜杜拉国王科技大学）
Deep networks trained on demonstrations of human driving have learned to follow roads and avoid obstacles. However, driving policies trained via imitation learning cannot be controlled at test time. A vehicle trained end-to-end to imitate an expert cannot be guided to take a specific turn at an upcoming intersection. This limits the utility of such systems. We propose to condition imitation learning on high-level command input. At test time, the learned driving policy functions as a chauffeur that handles sensorimotor coordination but continues to respond to navigational commands. We evaluate …
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
During retinal microsurgery, precise manipulation of the delicate retinal tissue is required for positive surgical outcome. However, accurate manipulation and navigation of surgical tools remain difficult due to a constrained workspace and the top-down view during the surgery, which limits the surgeon's ability to estimate depth. To alleviate such difficulty, we propose to automate the tool-navigation task by learning to predict relative goal position on the retinal surface from the current tool-tip position. Given an estimated target on the retina, we generate an optimal trajectory leading to…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Army Research Laboratory（陆军研究实验室）；Google（谷歌）
Social navigation is the capability of an autonomous agent, such as a robot, to navigate in a 'socially compliant' manner in the presence of other intelligent agents such as humans. With the emergence of autonomously navigating mobile robots in human populated environments (e.g., domestic service robots in homes and restaurants and food delivery robots on public sidewalks), incorporating socially compliant navigation behaviors on these robots becomes critical to ensuring safe and comfortable human robot coexistence. To address this challenge, imitation learning is a promising framework, since …
Sample-Efficient Learning with Online Expert Correction for Autonomous Catheter Steering in Endovascular Bifurcation Navigation
Tongji University（同济大学）；College of Electronics and Information Engineering（电子信息工程学院）；Shanghai Institute of Intelligent Science and Technology（上海智能科学与技术研究院）；The Chinese University of Hong Kong（香港中文大学）；School of Mechanical and Electric Engineering（机械电子工程学院）；Soochow University（苏州大学）；Shanghai Operation Robot Co., Ltd.（上海操作机器人有限公司）；State Key Laboratory of Cardiovascular Diseases（心血管疾病国家重点实验室）
Autonomous driving is a complex task, which has been tackled since the first self-driving car ALVINN in 1989, with a supervised learning approach, or behavioral cloning (BC). In BC, a neural network is trained with state-action pairs that constitute the training set made by an expert, i.e., a human driver. However, this type of imitation learning does not take into account the temporal dependencies that might exist between actions taken in different moments of a navigation trajectory. These type of tasks are better handled by reinforcement learning (RL) algorithms, which need to define a rewar…
Autonomous Navigation for Robot-assisted Intraluminal and Endovascular Procedures: A Systematic Review
University of Verona（维罗纳大学）；Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）；Politecnico di Milano（米兰理工大学）；Delft University of Technology（代尔夫特理工大学）；Scuola Superiore Sant’Anna（圣安娜高等研究院）；KU Leuven（鲁汶大学）
The ability to perform effective planning is crucial for building an instruction-following agent. When navigating through a new environment, an agent is challenged with (1) connecting the natural language instructions with its progressively growing knowledge of the world; and (2) performing long-range planning and decision making in the form of effective exploration and error correction. Current methods are still limited on both fronts despite extensive efforts. In this paper, we introduce the Evolving Graphical Planner (EGP), a model that performs global planning for navigation based on raw s…
A 2D Image Feedback-Based Autonomous Endovascular Navigation Method: A Perception and Control Framework
Beijing Institute of Technology（北京理工大学）；Advanced Institute for Ocean Research, Southern University of Science and Technology（南方科技大学海洋科学与工程研究院）；Shenzhen Institute of Advanced Biomedical Robot Company Limited（深圳先进生物医学机器人有限公司）；Southern University of Science and Technology（南方科技大学）；School of Mechanical, Electronic and Control Engineering, Beijing Jiaotong University（北京交通大学机械与电子控制工程学院）；Aerospace Center Hospital（航天中心医院）
In this paper we propose a number of tested ways in which a low-budget demo car could be made to navigate an indoor environment. Canny Edge Detection, Supervised Floor Detection and Imitation Learning were used separately and are contrasted in their effectiveness. The equipment used in this paper approximated an autonomous robot configured to work with a mobile device for image processing. This paper does not provide definitive solutions and simply illustrates the approaches taken to successfully achieve autonomous navigation of indoor environments. The successes and failures of all approaches…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Vanderbilt University（范德堡大学）
We tackle the problem of perceptive locomotion in dynamic environments. In this problem, a quadrupedal robot must exhibit robust and agile walking behaviors in response to environmental clutter and moving obstacles. We present a hierarchical learning framework, named PRELUDE, which decomposes the problem of perceptive locomotion into high-level decision-making to predict navigation commands and low-level gait generation to realize the target commands. In this framework, we train the high-level navigation controller with imitation learning on human demonstrations collected on a steerable cart a…
Asian Institute of Technology（亚洲理工学院）；The University of Tokyo（东京大学）
Compared to traditional imitation learning methods such as DAgger and DART, intervention-based imitation offers a more convenient and sample efficient data collection process to users. In this paper, we introduce Reinforced Intervention-based Learning (ReIL), a framework consisting of a general intervention-based learning algorithm and a multi-task imitation learning model aimed at enabling non-expert users to train agents in real environments with little supervision or fine tuning. ReIL achieves this with an algorithm that combines the advantages of imitation learning and reinforcement learni…
Learning to Turn: Diffusion Imitation for Robust Row Turning in Under-Canopy Robots
Field Robotics Engineering and Sciences Hub (FRESH), University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校野外机器人工程与科学中心（FRESH））；Department of Aerospace Engineering, University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校航空航天工程系）
Autonomous Guidewire Navigation for Robot-assisted Endovascular Interventions: A Knowledge-Driven Visual Guidance Approach
Wandong Medical Technology Co., Ltd（万东医疗科技有限公司）；Beijing Police College（北京警察学院）；School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications（北京邮电大学智能工程与自动化学院）；Department of Vascular Surgery, Shanghai Changhai Hospital, Naval Medical University（海军军医大学上海长海医院血管外科）；Beijing Institute of Satellite Information Engineering（北京卫星信息工程研究所）
Imitation learning aims to extract high-performance policies from logged demonstrations of expert behavior. It is common to frame imitation learning as a supervised learning problem in which one fits a function approximator to the input-output mapping exhibited by the logged demonstrations (input observations to output actions). While the framing of imitation learning as a supervised input-output learning problem allows for applicability in a wide variety of settings, it is also an overly simplistic view of the problem in situations where the expert demonstrations provide much richer insight i…
