This report demonstrates several methods used to make a self-driving vehicle using a supervised learning algorithm and a forward-facing RGBD camera. The project originally involved research in creating an adversarial attack on the vehicle's model, but due to difficulties with the initial training of the car, the plans were discarded in favor of completing the imitation learning portion of the project. Many approaches were explored, but due to challenges introduced by an unbalanced data set, the approaches had limited effectiveness.
The decision and planning system for autonomous driving in urban environments is hard to design. Most current methods manually design the driving policy, which can be expensive to develop and maintain at scale. Instead, with imitation learning we only need to collect data and the computer will learn and improve the driving policy automatically. However, existing imitation learning methods for autonomous driving are hardly performing well for complex urban scenarios. Moreover, the safety is not guaranteed when we use a deep neural network policy. In this paper, we proposed a framework to learn …
Institute for Robotics and Intelligent Machines（机器人与智能机器研究所）；School of Electrical and Computer Engineering（电气与计算机工程学院）；Georgia Institute of Technology（佐治亚理工学院）
We present an end-to-end imitation learning system for agile, off-road autonomous driving using only low-cost sensors. By imitating a model predictive controller equipped with advanced sensors, we train a deep neural network control policy to map raw, high-dimensional observations to continuous steering and throttle commands. Compared with recent approaches to similar tasks, our method requires neither state estimation nor on-the-fly planning to navigate the vehicle. Our approach relies on, and experimentally validates, recent imitation learning theory. Empirically, we show that policies train…
Two current methods used to train autonomous cars are reinforcement learning and imitation learning. This research develops a new learning methodology and systematic approach in both a simulated and a smaller real world environment by integrating supervised imitation learning into reinforcement learning to make the RL training data collection process more effective and efficient. By combining the two methods, the proposed research successfully leverages the advantages of both RL and IL methods. First, a real mini-scale robot car was assembled and trained on a 6 feet by 9 feet real world track …
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；University of Macau（澳门大学）
Existing imitation learning methods suffer from low efficiency and generalization ability when facing the road option problem in an urban environment. In this paper, we propose a yaw-guided imitation learning method to improve the road option performance in an end-to-end autonomous driving paradigm in terms of the efficiency of exploiting training samples and adaptability to changing environments. Specifically, the yaw information is provided by the trajectory of the navigation map. Our end-to-end architecture, Yaw-guided Imitation Learning with ResNet34 Attention (YILRatt), integrates the Res…
Seoul National University（首尔国立大学）；Dongguk University（东国大学）；Department of Electrical and Computer Engineering（电子与计算机工程系）；ASRI（ASRI（人工智能研究中心））；School of Computer Science and Engineering（计算机科学与工程学院）
In this paper, we consider the problem of autonomous driving using imitation learning in a semi-supervised manner. In particular, both labeled and unlabeled demonstrations are leveraged during training by estimating the quality of each unlabeled demonstration. If the provided demonstrations are corrupted and have a low signal-to-noise ratio, the performance of the imitation learning agent can be degraded significantly. To mitigate this problem, we propose a method called semi-supervised imitation learning (SSIL). SSIL first learns how to discriminate and evaluate each state-action pair's relia…
Advanced Institutes of Convergence Technology (AICT)（先进融合技术研究院（AICT））
Unstructured environments are difficult for autonomous driving. This is because various unknown obstacles are lied in drivable space without lanes, and its width and curvature change widely. In such complex environments, searching for a path in real-time is difficult. Also, inaccurate localization data reduce the path tracking accuracy, increasing the risk of collision. Instead of searching and tracking the path, an alternative approach has been proposed that reactively avoids obstacles in real-time. Some methods are available for tracking global path while avoiding obstacles using the candida…
In recent years, great efforts have been devoted to deep imitation learning for autonomous driving control, where raw sensory inputs are directly mapped to control actions. However, navigating through densely populated intersections remains a challenging task due to uncertainty caused by uncertain traffic participants. We focus on autonomous navigation at crowded intersections that require interaction with pedestrians. A multi-task conditional imitation learning framework is proposed to adapt both lateral and longitudinal control tasks for safe and efficient interaction. A new benchmark called…
iCOIL: Scenario Aware Autonomous Parking Via Integrated Constrained Optimization and Imitation Learning
Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；University of Hong Kong（香港大学）；University of Macau（澳门大学）
One of the challenges to reduce the gap between the machine and the human level driving is how to endow the system with the learning capacity to deal with the coupled complexity of environments, intentions, and dynamics. In this paper, we propose a hierarchical driving model with explicit model of continuous intention and continuous dynamics, which decouples the complexity in the observation-to-action reasoning in the human driving data. Specifically, the continuous intention module takes the route planning map obtained by GPS and IMU, perception from a RGB camera and LiDAR as input to generat…
We present a learning-based planner that aims to robustly drive a vehicle by mimicking human drivers' driving behavior. We leverage a mid-to-mid approach that allows us to manipulate the input to our imitation learning network freely. With that in mind, we propose a novel feedback synthesizer for data augmentation. It allows our agent to gain more driving experience in various previously unseen environments that are likely to encounter, thus improving overall performance. This is in contrast to prior works that rely purely on random synthesizers. Furthermore, rather than completely commit to i…
Northwestern University（西北大学）；Shirley Ryan AbilityLab（雪莉·瑞恩能力实验室）
We describe a shared control methodology that can, without knowledge of the task, be used to improve a human's control of a dynamic system, be used as a training mechanism, and be used in conjunction with Imitation Learning to generate autonomous policies that recreate novel behaviors. Our algorithm introduces autonomy that assists the human partner by enforcing safety and stability constraints. The autonomous agent has no a priori knowledge of the desired task and therefore only adds control information when there is concern for the safety of the system. We evaluate the efficacy of our approa…
Intel Labs（英特尔实验室）；Computer Vision Center & Univ. Autonoma de Barcelona（计算机视觉中心与巴塞罗那自治大学）；King Abdullah University of Science & Technology（阿卜杜拉国王科技大学）
Deep networks trained on demonstrations of human driving have learned to follow roads and avoid obstacles. However, driving policies trained via imitation learning cannot be controlled at test time. A vehicle trained end-to-end to imitate an expert cannot be guided to take a specific turn at an upcoming intersection. This limits the utility of such systems. We propose to condition imitation learning on high-level command input. At test time, the learned driving policy functions as a chauffeur that handles sensorimotor coordination but continues to respond to navigational commands. We evaluate …
Symbolic Imitation Learning: From Black-Box to Explainable Driving Policies
Department of Science and Engineering, the George Washington University（科学与工程系，乔治·华盛顿大学）；Autonomous Vehicles Lab, Department of Mechanical Engineering, University of Surrey（自主车辆实验室，机械工程系，萨里大学）
Phase-Conditioned Imitation Learning with Autonomous Failure Recovery for Robust Deformable Object Manipulation
Department of Robotics, Tohoku University（东大理学院机器人系）；JC STEM Lab of Robotics for Soft Materials, the Department of Electrical and Computer Engineering, Faculty of Engineering, The University of Hong Kong（香港大学工程学院电气与计算机工程系软材料机器人实验室）
With the evolution of various advanced driver assistance system (ADAS) platforms, the design of autonomous driving system is becoming more complex and safety-critical. The autonomous driving system simultaneously activates multiple ADAS functions; and thus it is essential to coordinate various ADAS functions. This paper proposes a randomized adversarial imitation learning (RAIL) method that imitates the coordination of autonomous vehicle equipped with advanced sensors. The RAIL policies are trained through derivative-free optimization for the decision maker that coordinates the proper ADAS fun…
SRM Institute of Science and Technology（SRM科学技术研究所）
In this work, we present a rigorous end-to-end control strategy for autonomous vehicles aimed at minimizing lap times in a time attack racing event. We also introduce AutoRACE Simulator developed as a part of this research project, which was employed to simulate accurate vehicular and environmental dynamics along with realistic audio-visual effects. We adopted a hybrid imitation-reinforcement learning architecture and crafted a novel reward function to train a deep neural network policy to drive (using imitation learning) and race (using reinforcement learning) a car autonomously in less than …
MRIC: Model-Based Reinforcement-Imitation Learning with Mixture-of-Codebooks for Autonomous Driving Simulation
State Key Laboratory of Intelligent Green Vehicle and Mobility, School of Vehicle and Mobility, Tsinghua University（清华大学车辆与 Mobility学院智能绿色车辆与交通全国重点实验室）
Distortion-Resilient Robotic Imitation Learning for Autonomous Cable Routing
School of Artificial Intelligence, Harbin Institute of Technology（哈尔滨工业大学人工智能学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；Pengcheng Laboratory（鹏城实验室）；Suzhou Research Institute, Harbin Institute of Technology（哈尔滨工业大学苏州研究院）
Five AI Ltd.（Five AI有限公司）；University of Oxford（牛津大学）；University of Edinburgh（爱丁堡大学）
Achieving a proper balance between planning quality, safety and efficiency is a major challenge for autonomous driving. Optimisation-based motion planners are capable of producing safe, smooth and comfortable plans, but often at the cost of runtime efficiency. On the other hand, naively deploying trajectories produced by efficient-to-run deep imitation learning approaches might risk compromising safety. In this paper, we present PILOT -- a planning framework that comprises an imitation neural network followed by an efficient optimiser that actively rectifies the network's plan, guaranteeing fu…
UC Berkeley（加州大学伯克利分校）；Robotics at Google（谷歌机器人）；Stanford（斯坦福大学）
While robot learning has demonstrated promising results for enabling robots to automatically acquire new skills, a critical challenge in deploying learning-based systems is scale: acquiring enough data for the robot to effectively generalize broadly. Imitation learning, in particular, has remained a stable and powerful approach for robot learning, but critically relies on expert operators for data collection. In this work, we target this challenge, aiming to build an imitation learning system that can continuously improve through autonomous data collection, while simultaneously avoiding the ex…
RCM-ACT: Imitation Learning with Dynamic RCM Calibration for Autonomous Intraocular Foreign Body Removal
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；State Key Laboratory of Transvascular Implantation Devices of the Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院第二附属医院血管植入设备国家重点实验室）；Dessight Biomedical（Dessight生物医学公司）；Center for Rehabilitation Medicine, Department of Ophthalmology, Zhejiang Provincial People’s Hospital（浙江省人民医院康复医学中心、眼科部门）；School of Biosystems Engineering and Food Science, Zhejiang University（浙江大学生物系统工程与食品科学学院）；School of Public Health and Second Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院公共卫生学院及第二附属医院）；State Key Laboratory of Transvascular Implantation Devices of the Second Affiliated Hospital and School of Public Health, Zhejiang University School of Medicine（浙江大学医学院第二附属医院及公共卫生学院血管植入设备国家重点实验室）；Zhejiang Key Laboratory of Medical Imaging Artificial Intelligence（浙江省医学影像人工智能重点实验室）
Hand-crafting generalised decision-making rules for real-world urban autonomous driving is hard. Alternatively, learning behaviour from easy-to-collect human driving demonstrations is appealing. Prior work has studied imitation learning (IL) for autonomous driving with a number of limitations. Examples include only performing lane-following rather than following a user-defined route, only using a single camera view or heavily cropped frames lacking state observability, only lateral (steering) control, but not longitudinal (speed) control and a lack of interaction with traffic. Importantly, the…
An accurate model of the environment and the dynamic agents acting in it offers great potential for improving motion planning. We present MILE: a Model-based Imitation LEarning approach to jointly learn a model of the world and a policy for autonomous driving. Our method leverages 3D geometry as an inductive bias and learns a highly compact latent space directly from high-resolution videos of expert demonstrations. Our model is trained on an offline corpus of urban driving data, without any online interaction with the environment. MILE improves upon prior state-of-the-art by 31% in driving sco…
JetBrains Research（JetBrains研究院）；National Research University Higher School of Economics（俄罗斯国家研究型高等经济大学）
In this paper, we describe our winning approach to solving the Lane Following Challenge at the AI Driving Olympics Competition through imitation learning on a mixed set of simulation and real-world data. AI Driving Olympics is a two-stage competition: at stage one, algorithms compete in a simulated environment with the best ones advancing to a real-world final. One of the main problems that participants encounter during the competition is that algorithms trained for the best performance in simulated environments do not hold up in a real-world environment and vice versa. Classic control algorit…
Siemens Digital Industries Software（西门子数字工业软件）；KU Leuven（鲁汶大学）；Flanders Make（法兰德斯制造研究院）
To ensure user acceptance of autonomous vehicles (AVs), control systems are being developed to mimic human drivers from demonstrations of desired driving behaviors. Imitation learning (IL) algorithms serve this purpose, but struggle to provide safety guarantees on the resulting closed-loop system trajectories. On the other hand, Model Predictive Control (MPC) can handle nonlinear systems with safety constraints, but realizing human-like driving with it requires extensive domain knowledge. This work suggests the use of a seamless combination of the two techniques to learn safe AV controllers fr…
The American University in Cairo（开罗美国大学）；Valeo Schalter und Sensoren GmbH（法雷奥开关与传感器有限公司）；Valeo Egypt（法雷奥埃及公司）
Conditional imitation learning (CIL) trains deep neural networks, in an end-to-end manner, to mimic human driving. This approach has demonstrated suitable vehicle control when following roads, avoiding obstacles, or taking specific turns at intersections to reach a destination. Unfortunately, performance dramatically decreases when deployed to unseen environments and is inconsistent against varying weather conditions. Most importantly, the current CIL fails to avoid static road blockages. In this work, we propose a solution to those deficiencies. First, we fuse the laser scanner with the regul…
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Army Research Laboratory（陆军研究实验室）；Sony AI（索尼人工智能）
While imitation learning for vision based autonomous mobile robot navigation has recently received a great deal of attention in the research community, existing approaches typically require state action demonstrations that were gathered using the deployment platform. However, what if one cannot easily outfit their platform to record these demonstration signals or worse yet the demonstrator does not have access to the platform at all? Is imitation learning for vision based autonomous navigation even possible in such scenarios? In this work, we hypothesize that the answer is yes and that recent …
As the advanced driver assistance system (ADAS) functions become more sophisticated, the strategies that properly coordinate interaction and communication among the ADAS functions are required for autonomous driving. This paper proposes a derivative-free optimization based imitation learning method for the decision maker that coordinates the proper ADAS functions. The proposed method is able to make decisions in multi-lane highways timely with the LIDAR data. The simulation-based evaluation verifies that the proposed method presents desired performance.
Mercedes-Benz AG R&D（梅赛德斯-奔驰股份公司研发部）；University of Tübingen（蒂宾根大学）；Max Planck Institute for Intelligent Systems（马克斯·普朗克智能系统研究所）
Simulators offer the possibility of safe, low-cost development of self-driving systems. However, current driving simulators exhibit naïve behavior models for background traffic. Hand-tuned scenarios are typically added during simulation to induce safety-critical situations. An alternative approach is to adversarially perturb the background traffic trajectories. In this paper, we study this approach to safety-critical driving scenario generation using the CARLA simulator. We use a kinematic bicycle model as a proxy to the simulator's true dynamics and observe that gradients through this proxy m…
We demonstrate the first large-scale application of model-based generative adversarial imitation learning (MGAIL) to the task of dense urban self-driving. We augment standard MGAIL using a hierarchical model to enable generalization to arbitrary goal routes, and measure performance using a closed-loop evaluation framework with simulated interactive agents. We train policies from expert trajectories collected from real vehicles driving over 100,000 miles in San Francisco, and demonstrate a steerable policy that can navigate robustly even in a zero-shot setting, generalizing to synthetic scenari…
Johns Hopkins University（约翰斯·霍普金斯大学）；Johns Hopkins University School of Medicine（约翰斯·霍普金斯大学医学院）
During retinal microsurgery, precise manipulation of the delicate retinal tissue is required for positive surgical outcome. However, accurate manipulation and navigation of surgical tools remain difficult due to a constrained workspace and the top-down view during the surgery, which limits the surgeon's ability to estimate depth. To alleviate such difficulty, we propose to automate the tool-navigation task by learning to predict relative goal position on the retinal surface from the current tool-tip position. Given an estimated target on the retina, we generate an optimal trajectory leading to…
Imitation learning is a promising approach to end-to-end training of autonomous vehicle controllers. Typically the driving process with such approaches is entirely automatic and black-box, although in practice it is desirable to control the vehicle through high-level commands, such as telling it which way to go at an intersection. In existing work this has been accomplished by the application of a branched neural architecture, since directly providing the command as an additional input to the controller often results in the command being ignored. In this work we overcome this limitation by lea…
Tsinghua University（清华大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；University of Pennsylvania（宾夕法尼亚大学）；Shanghai Qi Zhi Institute（上海期智研究院）
Imitation learning trains control policies by mimicking pre-recorded expert demonstrations. In partially observable settings, imitation policies must rely on observation histories, but many seemingly paradoxical results show better performance for policies that only access the most recent observation. Recent solutions ranging from causal graph learning to deep information bottlenecks have shown promising results, but failed to scale to realistic settings such as visual imitation. We propose a solution that outperforms these prior approaches by upweighting demonstration keyframes corresponding …
SSIL: Self-Supervised Imitation Learning for End-to-End Driving
R & D Center, SL Corporation（SL公司研发中心）；Department of Electrical and Computer Engineering, Sungkyunkwan University（成均馆大学电子与计算机工程系）；Department of Information and Communication Engineering, Handong Global University（翰林全球大学信息与通信工程系）；College of Engineering, Texas A & M University-Corpus Christi（德克萨斯A&M大学科珀斯克里斯蒂分校工程学院）；School of Computer Science and Electrical Engineering, Handong Global University（翰林全球大学计算机科学与电子工程学院）
Prompting Multi-Modal Tokens to Enhance End-to-End Autonomous Driving Imitation Learning with LLMs
University of Technology Sydney（悉尼科技大学）；Australia Artificial Intelligence Institute（澳大利亚人工智能研究院）；School of Computer Science（计算机学院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
One way to approach end-to-end autonomous driving is to learn a policy function that maps from a sensory input, such as an image frame from a front-facing camera, to a driving action, by imitating an expert driver, or a reference policy. This can be done by supervised learning, where a policy function is tuned to minimize the difference between the predicted and ground-truth actions. A policy function trained in this way however is known to suffer from unexpected behaviours due to the mismatch between the states reachable by the reference policy and trained policy functions. More advanced algo…
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）
Detection of road curbs is an essential capability for autonomous driving. It can be used for autonomous vehicles to determine drivable areas on roads. Usually, road curbs are detected on-line using vehicle-mounted sensors, such as video cameras and 3-D Lidars. However, on-line detection using video cameras may suffer from challenging illumination conditions, and Lidar-based approaches may be difficult to detect far-away road curbs due to the sparsity issue of point clouds. In recent years, aerial images are becoming more and more worldwide available. We find that the visual appearances betwee…
Toward Autonomous Soft Robotic Endovascular Navigation via Imitation Learning
Johns Hopkins University（约翰霍普金斯大学）；Stanford University（斯坦福大学）；McGill University（麦吉尔大学）；University of Maryland（马里兰大学）；Swiss Federal Institute of Technology in Lausanne (EPFL)（日内瓦联邦理工学院（EPFL））；ETH Zurich（苏黎世联邦理工学院）
Autonomous driving is a complex task, which has been tackled since the first self-driving car ALVINN in 1989, with a supervised learning approach, or behavioral cloning (BC). In BC, a neural network is trained with state-action pairs that constitute the training set made by an expert, i.e., a human driver. However, this type of imitation learning does not take into account the temporal dependencies that might exist between actions taken in different moments of a navigation trajectory. These type of tasks are better handled by reinforcement learning (RL) algorithms, which need to define a rewar…
