Reinforcement learning (RL) has shown to be a valuable tool in training neural networks for autonomous motion planning. The application of RL to a specific problem is dependent on a reward signal to quantify how good or bad a certain action is. This paper addresses the problem of reward signal design for robotic control in the context of local planning for autonomous racing. We aim to design reward signals that are able to perform well in multiple, competing, continuous metrics. Three different methodologies of position-based, velocity-based, and action-based rewards are considered and evaluated in the context of F1/10th racing. A novel method of rewarding the agent on its state relative to an optimal trajectory is presented. Agents are trained and tested in simulation and the behaviors generated by the reward signals are compared to each other on the basis of average lap time and completion rate. The results indicate that a reward based on the distance and velocity relative to a minimum curvature trajectory produces the fastest lap times.
Georgia Institute of Technology（佐治亚理工学院）；Twentieth Century Fox（二十世纪福克斯）；Starfleet Academy（星际舰队学院）；Tyrell Inc.（泰瑞尔公司）
A fundamental aspect of racing is overtaking other race cars. Whereas previous research on autonomous racing has majorly focused on lap-time optimization, here, we propose a method to plan overtaking maneuvers in autonomous racing. A Gaussian process is used to learn the behavior of the leading vehicle. Based on the outputs of the Gaussian process, a stochastic Model Predictive Control algorithm plans optimistic trajectories, such that the controlled autonomous race car is able to overtake the leading vehicle. The proposed method is tested in a simple simulation scenario.
Korea Advanced Institute of Science and Technology（韩国科学技术院）
Resolving edge-cases in autonomous driving, head-to-head autonomous racing is getting a lot of attention from the industry and academia. In this study, we propose a game-theoretic model predictive control (MPC) approach for head-to-head autonomous racing and data-driven model identification method. For the practical estimation of nonlinear model parameters, we adopted the hyperband algorithm, which is used for neural model training in machine learning. The proposed controller comprises three modules: 1) game-based opponents' trajectory predictor, 2) high-level race strategy planner, and 3) MPC…
Robust Spatiotemporal Motion Planning for Multi-Agent Autonomous Racing via Topological Gap Identification and Accelerated MPC
Department of Control Science and Engineering, Zhejiang University（浙江大学控制科学与工程学院）；Center for Project-Based Learning, D-ITET, ETH Zurich（苏黎世联邦理工学院项目导向学习中心）
Stony Brook University（石溪大学）；University of Pennsylvania（宾夕法尼亚大学）
High-performance autonomy often must operate at the boundaries of safety. When external agents are present in a system, the process of ensuring safety without sacrificing performance becomes extremely difficult. In this paper, we present an approach to stress test such systems based on the rapidly exploring random tree (RRT) algorithm. We propose to find faults in such systems through adversarial agent perturbations, where the behaviors of other agents in an otherwise fixed scenario are modified. This creates a large search space of possibilities, which we explore both randomly and with a focu…
Formula Student Driverless challenges engineering students to develop autonomous single-seater race cars in a quest to bring about more graduates who are well-prepared to solve the real world problems associated with autonomous driving. In this paper, we present the software stack of KA-RaceIng's entry to the 2019 competitions. We cover the essential modules of the system, including perception, localization, mapping, motion planning, and control. Furthermore, development methods are outlined and an overview of the system architecture is given. We conclude by presenting selected runtime measure…
Online Velocity Profile Generation and Tracking for Sampling-Based Local Planning Algorithms in Autonomous Racing Environments
Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich（自主车辆系统教授职位，技术大学慕尼黑工程与设计学院，技术大学慕尼黑）；Chair of Automatic Control, Technical University of Munich（自动控制教授职位，技术大学慕尼黑）；Institute of Automotive Technology, TUM School of Engineering and Design, Technical University of Munich（汽车技术研究所，技术大学慕尼黑工程与设计学院，技术大学慕尼黑）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所（MIRMI））
University of Sannio（萨尼奥大学）；Czech Technical University in Prague（捷克布拉格技术大学）；Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）
A Nonlinear Model Predictive Control (NMPC) strategy aimed at controlling a small-scale car model for autonomous racing competitions is presented in this paper. The proposed control strategy is concerned with minimizing the lap time while keeping the vehicle within track boundaries. The optimization problem considers both the vehicle's actuation limits and the lateral and longitudinal forces acting on the car modeled through the Pacejka's magic formula and a simple drivetrain model. Furthermore, the approach allows to safely race on a track populated by static obstacles generating collision-fr…
RaceMOP: Mapless Online Path Planning for Multi-Agent Autonomous Racing using Residual Policy Learning
TUM School of Engineering and Design, Technical University of Munich（慕尼黑工业大学工程与设计学院）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）
A Hybrid Sampling-Based Trajectory Planner with Game-Theoretic Guidance for Autonomous Racing
Professorship of Autonomous Vehicle Systems, TUM School of Engineering and Design, Technical University of Munich（慕尼黑工业大学工程与设计学院自动驾驶车辆系统教授组）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）
Efficient Trajectory Optimization for Autonomous Racing via Formula-1 Data-Driven Initialization
University of Bonn（波恩大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）；Center for Robotics（机器人中心）；German Federal Ministry of Research, Technology and Space（德国联邦研究、科技与航天部）
The University of Manchester（曼彻斯特大学）；Oxford Brookes University（牛津布鲁克斯大学）
Widespread development of driverless vehicles has led to the formation of autonomous racing, where technological development is accelerated by the high speeds and competitive environment of motorsport. A particular challenge for an autonomous vehicle is that of identifying a target trajectory - or, in the case of a competition vehicle, the racing line. Many existing approaches to finding the racing line are either not time-optimal solutions, or are computationally expensive - rendering them unsuitable for real-time application using on-board processing hardware. This study describes a machine …
We present an approach for safe trajectory planning, where a strategic task related to autonomous racing is learned sample-efficient within a simulation environment. A high-level policy, represented as a neural network, outputs a reward specification that is used within the cost function of a parametric nonlinear model predictive controller (NMPC). By including constraints and vehicle kinematics in the NLP, we are able to guarantee safe and feasible trajectories related to the used model. Compared to classical reinforcement learning (RL), our approach restricts the exploration to safe trajecto…
Regulation-Aware Game-Theoretic Motion Planning for Autonomous Racing
University of Modena and Reggio Emilia（摩德纳和雷吉奥艾米利亚大学）；University of Bologna（博洛尼亚大学）；University of California, Berkeley（加州大学伯克利分校）；Chalmers University of Technology（查尔姆斯理工大学）
Munich Electrification（慕尼黑电气化公司）；Arrival（Arrival公司）；The University of Edinburgh（爱丁堡大学）
Accurately modeling robot dynamics is crucial to safe and efficient motion control. In this paper, we develop and apply an iterative learning semi-parametric model, with a neural network, to the task of autonomous racing with a Model Predictive Controller (MPC). We present a novel non-linear semi-parametric dynamics model where we represent the known dynamics with a parametric model, and a neural network captures the unknown dynamics. We show that our model can learn more accurately than a purely parametric model and generalize better than a purely non-parametric model, making it ideal for rea…
In this paper we propose a hierarchical controller for autonomous racing where the same vehicle model is used in a two level optimization framework for motion planning. The high-level controller computes a trajectory that minimizes the lap time, and the low-level nonlinear model predictive path following controller tracks the computed trajectory online. Following a computed optimal trajectory avoids online planning and enables fast computational times. The efficiency is further enhanced by the coupling of the two levels through a terminal constraint, computed in the high-level controller. Incl…
The rising popularity of driver-less cars has led to the research and development in the field of autonomous racing, and overtaking in autonomous racing is a challenging task. Vehicles have to detect and operate at the limits of dynamic handling and decisions in the car have to be made at high speeds and high acceleration. One of the most crucial parts in autonomous racing is path planning and decision making for an overtaking maneuver with a dynamic opponent vehicle. In this paper we present the evaluation of a track based offline policy learning approach for autonomous racing. We define spec…
Hamburg University of Technology（汉堡工业大学）；UC Berkeley（加州大学伯克利分校）
The goal of this thesis is to design a learning model predictive controller (LMPC) that allows multiple agents to race competitively on a predefined race track in real-time. This thesis addresses two major shortcomings in the already existing single-agent formulation. Previously, the agent determines a locally optimal trajectory but does not explore the state space, which may be necessary for overtaking maneuvers. Additionally, obstacle avoidance for LMPC has been achieved in the past by using a non-convex terminal set, which increases the complexity for determining a solution to the optimizat…
MCTR: Midpoint Corrected Triangulation for Autonomous Racing via Digital Twin Simulation in CARLA
Department of Control Science and Engineering, Zhejiang University, Hangzhou, China（浙江大学控制科学与工程学院）；Ningbo Innovation Center, Zhejiang University, Ningbo, China（浙江大学宁波创新中心）；Center for Project-Based Learning, D-ITET, ETH Zurich（苏黎世联邦理工学院项目导向学习中心）；Technische Universität Wien, Vienna, Austria（维也纳技术大学）
本文提出 MCTR 算法，通过曲率校正移动平均改进轨迹平滑性，并在 CARLA 数字孪生中验证 3D LiDAR 感知下的自主赛车性能。
Fast and Modular Autonomy Software for Autonomous Racing Vehicles
Carnegie Mellon University（卡内基梅隆大学）；Cornell University（康奈尔大学）；Massachusetts Institute of Technology（麻省理工学院）；University of Pittsburgh（匹兹堡大学）；Rochester Institute of Technology（罗切斯特理工学院）；University of Waterloo（滑铁卢大学）
Benchmarking Empirical and Learning-Based Approaches for Feedforward Steering Control in Autonomous Racing
Chair of Automatic Control, Department of Engineering Physics and Computation, Technical University of Munich（慕尼黑技术大学自动控制系）；Professorship of Autonomous Vehicle Systems (AVS), Department of Mobility Systems Engineering, Technical University of Munich（慕尼黑技术大学移动系统工程系自动驾驶车辆系统教授职位）
This paper presents an adaptive lookahead pure-pursuit lateral controller for optimizing racing metrics such as lap time, average lap speed, and deviation from a reference trajectory in an autonomous racing scenario. We propose a greedy algorithm to compute and assign optimal lookahead distances for the pure-pursuit controller for each waypoint on a reference trajectory for improving the race metrics. We use a ROS based autonomous racing simulator to evaluate the adaptive pure-pursuit algorithm and compare our method with several other pure-pursuit based lateral controllers. We also demonstrat…
Zhejiang University（浙江大学）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）；DJI（大疆）
With the autonomy of aerial robots advances in recent years, autonomous drone racing has drawn increasing attention. In a professional pilot competition, a skilled operator always controls the drone to agilely avoid obstacles in aggressive attitudes, for reaching the destination as fast as possible. Autonomous flight like elite pilots requires planning in SE(3), whose non-triviality and complexity hindering a convincing solution in our community by now. To bridge this gap, this paper proposes an open-source baseline, which includes a high-performance SE(3) planner and a challenging simulation …
Drone racing is becoming a popular e-sport all over the world, and beating the best human drone race pilots has quickly become a new major challenge for artificial intelligence and robotics. In this paper, we propose a strategy for autonomous drone racing which is computationally more efficient than navigation methods like visual inertial odometry and simultaneous localization and mapping. This fast light-weight vision-based navigation algorithm estimates the position of the drone by fusing race gate detections with model dynamics predictions. Theoretical analysis and simulation results show t…
Indian Institute of Technology, Kharagpur（印度理工学院卡拉格普尔分校）；IUPUI（印第安纳大学-普渡大学印第安纳波利斯联合分校）
The paper presents a strategy for the control of anautonomous racing car on a pre-mapped track. Using a dynamic model of the vehicle, the optimal racing line is computed, taking track boundaries into account. With the optimal racing line as areference, a local nonlinear model predictive controller (NMPC) is proposed, which takes into account multiple local objectives like making more progress along the race line, avoiding collision with opponent vehicles, and use of drafting to achieve more progress.
Event-based vision has already revolutionized the perception task for robots by promising faster response, lower energy consumption, and lower bandwidth without introducing motion blur. In this work, a novel deep learning method based on gated recurrent units utilizing sparse convolutions for detecting gates in a race track is proposed using event-based vision for the autonomous drone racing problem. We demonstrate the efficiency and efficacy of the perception pipeline on a real robot platform that can safely navigate a typical autonomous drone racing track in real-time. Throughout the experim…
Expert Knowledge-driven Reinforcement Learning for Autonomous Racing via Trajectory Guidance and Dynamics Constraints
College of Automotive and Energy Engineering, Tongji University（同济大学汽车与能源工程学院）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与航空航天工程学院）
SPARK: Low Latency Single-Camera 3D Pose Estimation for Autonomous Racing using Keypoints
Technical University of Munich（慕尼黑工业大学）；School of Engineering & Design, Department of Mobility Systems Engineering, Institute of Automotive Technology（工程与设计学院，移动系统工程系，汽车技术研究所）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）
Head-to-Head autonomous racing at the limits of handling in the A2RL challenge
Institute of Automotive Technology, TUM（汽车技术研究所，慕尼黑技术大学）；Chair of Automatic Control, TUM（自动控制教授座，慕尼黑技术大学）；Professorship Autonomous Vehicle Systems, TUM（自主车辆系统教授职位，慕尼黑技术大学）
