The way to full autonomy of public road vehicles requires the step-by-step replacement of the human driver, with the ultimate goal of replacing the driver completely. Eventually, the driving software has to be able to handle all situations that occur on its own, even emergency situations. These particular situations require extreme combined braking and steering actions at the limits of handling to avoid an accident or to diminish its consequences. An average human driver is not trained to handle such extreme and rarely occurring situations and therefore often fails to do so. However, professional race drivers are trained to drive a vehicle utilizing the maximum amount of possible tire forces. These abilities are of high interest for the development of autonomous driving software. Here, we compare a professional race driver and our software stack developed for autonomous racing with data analysis techniques established in motorsports. The goal of this research is to derive indications for further improvement of the performance of our software and to identify areas where it still fails to meet the performance level of the human race driver. Our results are used to extend our software's capabilities and also to incorporate our findings into the research and development of public road autonomous vehicles.
First-person view drone racing has become a popular televised sport. However, very little is known about the perceptual and motor skills of professional drone racing pilots. A better understanding of these skills may inform path planning and control algorithms for autonomous multirotor flight. By using a real-world drone racing track and a large-scale position tracking system, we compare the drone racing performance of five professional and five beginner pilots. Results show that professional pilots consistently outperform beginner pilots by achieving faster lap times, higher velocity, and mor…
Existing research on autonomous driving primarily focuses on urban driving, which is insufficient for characterising the complex driving behaviour underlying high-speed racing. At the same time, existing racing simulation frameworks struggle in capturing realism, with respect to visual rendering, vehicular dynamics, and task objectives, inhibiting the transfer of learning agents to real-world contexts. We introduce a new environment, where agents Learn-to-Race (L2R) in simulated competition-style racing, using multimodal information--from virtual cameras to a comprehensive array of inertial me…
Accelerating Autonomy: Insights from Pro Racers in the Era of Autonomous Racing - An Expert Interview Study
Technical University of Munich（慕尼黑工业大学）；TUM School of Engineering and Design（慕尼黑工业大学工程与设计学院）；Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）
This paper describes Ariel Team's autonomous racing controller for the Indy Autonomous Challenge (IAC) simulation race. IAC is the first multi-vehicle autonomous head-to-head competition, reaching speeds of 300 km/h along an oval track, modeled after the Indianapolis Motor Speedway (IMS). Our racing controller attempts to maximize progress along the track while avoiding collisions with opponent vehicles and obeying the race rules. To this end, the racing controller first computes a race line offline. Then, it repeatedly computes online a small set of dynamically feasible maneuver candidates, e…
Robotics is the next frontier in the progress of Artificial Intelligence (AI), as the real world in which robots operate represents an enormous, complex, continuous state space with inherent real-time requirements. One extreme challenge in robotics is currently formed by autonomous drone racing. Human drone racers can fly through complex tracks at speeds of up to 190 km/h. Achieving similar speeds with autonomous drones signifies tackling fundamental problems in AI under extreme restrictions in terms of resources. In this article, we present the winning solution of the first AI Robotic Racing …
Carnegie Mellon University（卡内基梅隆大学）；Bosch Research Pittsburgh（博世匹兹堡研究院）；NVIDIA（英伟达）；AICrowd；Amazon（亚马逊）；Arrival Ltd.（Arrival公司）；Pinterest
We present the results of our autonomous racing virtual challenge, based on the newly-released Learn-to-Race (L2R) simulation framework, which seeks to encourage interdisciplinary research in autonomous driving and to help advance the state of the art on a realistic benchmark. Analogous to racing being used to test cutting-edge vehicles, we envision autonomous racing to serve as a particularly challenging proving ground for autonomous agents as: (i) they need to make sub-second, safety-critical decisions in a complex, fast-changing environment; and (ii) both perception and control must be robu…
Drone racing is becoming a popular e-sport all over the world, and beating the best human drone race pilots has quickly become a new major challenge for artificial intelligence and robotics. In this paper, we propose a strategy for autonomous drone racing which is computationally more efficient than navigation methods like visual inertial odometry and simultaneous localization and mapping. This fast light-weight vision-based navigation algorithm estimates the position of the drone by fusing race gate detections with model dynamics predictions. Theoretical analysis and simulation results show t…
This paper presents a novel system for autonomous, vision-based drone racing combining learned data abstraction, nonlinear filtering, and time-optimal trajectory planning. The system has successfully been deployed at the first autonomous drone racing world championship: the 2019 AlphaPilot Challenge. Contrary to traditional drone racing systems, which only detect the next gate, our approach makes use of any visible gate and takes advantage of multiple, simultaneous gate detections to compensate for drift in the state estimate and build a global map of the gates. The global map and drift-compen…
Humans race drones faster than neural networks trained for end-to-end autonomous flight. This may be related to the ability of human pilots to select task-relevant visual information effectively. This work investigates whether neural networks capable of imitating human eye gaze behavior and attention can improve neural network performance for the challenging task of vision-based autonomous drone racing. We hypothesize that gaze-based attention prediction can be an efficient mechanism for visual information selection and decision making in a simulator-based drone racing task. We test this hypot…
While the majority of autonomous driving research has concentrated on everyday driving scenarios, further safety and performance improvements of autonomous vehicles require a focus on extreme driving conditions. In this context, autonomous racing is a new area of research that has been attracting considerable interest recently. Due to the fact that a vehicle is driven by its perception, planning, and control limits during racing, numerous research and development issues arise. This paper provides a comprehensive overview of the autonomous racing system built by team KAIST for the Indy Autonomo…
University of Pennsylvania（宾夕法尼亚大学）；ETH Zurich（苏黎世联邦理工学院）；California Institute of Technology（加州理工学院）；Technical University of Munich（慕尼黑工业大学）；University of Virginia（弗吉尼亚大学）；Clemson University（克莱姆森大学）
The rising popularity of self-driving cars has led to the emergence of a new research field in the recent years: Autonomous racing. Researchers are developing software and hardware for high performance race vehicles which aim to operate autonomously on the edge of the vehicles limits: High speeds, high accelerations, low reaction times, highly uncertain, dynamic and adversarial environments. This paper represents the first holistic survey that covers the research in the field of autonomous racing. We focus on the field of autonomous racecars only and display the algorithms, methods and approac…
Environment as Policy: Learning to Race in Unseen Tracks
Robotics and Perception Group, Department of Informatics, University of Zurich（机器人感知组，信息学院，苏黎世大学）；Department of Neuroinformatics, University of Zurich and ETH Zurich（神经信息学院，苏黎世大学和苏黎世联邦理工学院）
Event-based vision has already revolutionized the perception task for robots by promising faster response, lower energy consumption, and lower bandwidth without introducing motion blur. In this work, a novel deep learning method based on gated recurrent units utilizing sparse convolutions for detecting gates in a race track is proposed using event-based vision for the autonomous drone racing problem. We demonstrate the efficiency and efficacy of the perception pipeline on a real robot platform that can safely navigate a typical autonomous drone racing track in real-time. Throughout the experim…
Microsoft（微软）；Stanford University（斯坦福大学）；University of Zurich and ETH（苏黎世大学和苏黎世联邦理工学院）
Autonomous drone racing is a challenging research problem at the intersection of computer vision, planning, state estimation, and control. We introduce AirSim Drone Racing Lab, a simulation framework for enabling fast prototyping of algorithms for autonomy and enabling machine learning research in this domain, with the goal of reducing the time, money, and risks associated with field robotics. Our framework enables generation of racing tracks in multiple photo-realistic environments, orchestration of drone races, comes with a suite of gate assets, allows for multiple sensor modalities (monocul…
Zhejiang University（浙江大学）；Huzhou Institute of Zhejiang University（浙江大学湖州研究院）；DJI（大疆）
With the autonomy of aerial robots advances in recent years, autonomous drone racing has drawn increasing attention. In a professional pilot competition, a skilled operator always controls the drone to agilely avoid obstacles in aggressive attitudes, for reaching the destination as fast as possible. Autonomous flight like elite pilots requires planning in SE(3), whose non-triviality and complexity hindering a convincing solution in our community by now. To bridge this gap, this paper proposes an open-source baseline, which includes a high-performance SE(3) planner and a challenging simulation …
University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Intel（英特尔）
Autonomous micro aerial vehicles still struggle with fast and agile maneuvers, dynamic environments, imperfect sensing, and state estimation drift. Autonomous drone racing brings these challenges to the fore. Human pilots can fly a previously unseen track after a handful of practice runs. In contrast, state-of-the-art autonomous navigation algorithms require either a precise metric map of the environment or a large amount of training data collected in the track of interest. To bridge this gap, we propose an approach that can fly a new track in a previously unseen environment without a precise …
University of Pennsylvania, School of Engineering and Applied Science（宾夕法尼亚大学工程与应用科学学院）
The 3rd Japan Automotive AI Challenge was an international online autonomous racing challenge where 164 teams competed in December 2021. This paper outlines the winning strategy to this competition, and the advantages and challenges of using the Autoware.Auto open source autonomous driving platform for multi-agent racing. Our winning approach includes a lane-switching opponent overtaking strategy, a global raceline optimization, and the integration of various tools from Autoware.Auto including a Model-Predictive Controller. We describe the use of perception, planning and control modules for hi…
Stanford University（斯坦福大学）；University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Universidad de Zaragoza（萨拉戈萨大学）
To be successful in multi-player drone racing, a player must not only follow the race track in an optimal way, but also compete with other drones through strategic blocking, faking, and opportunistic passing while avoiding collisions. Since unveiling one's own strategy to the adversaries is not desirable, this requires each player to independently predict the other players' future actions. Nash equilibria are a powerful tool to model this and similar multi-agent coordination problems in which the absence of communication impedes full coordination between the agents. In this paper, we propose a…
Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles
School of Automation and Perception, Shanghai Jiao Tong University（自动化与感知学院，上海交通大学）；Faculty of Aerospace Engineering, Delft University of Technology（航空航天工程学院，代尔夫特理工大学）
Humans race drones faster than algorithms, despite being limited to a fixed camera angle, body rate control, and response latencies in the order of hundreds of milliseconds. A better understanding of the ability of human pilots of selecting appropriate motor commands from highly dynamic visual information may provide key insights for solving current challenges in vision-based autonomous navigation. This paper investigates the relationship between human eye movements, control behavior, and flight performance in a drone racing task. We collected a multimodal dataset from 21 experienced drone pil…
Technical University of Munich（慕尼黑工业大学）；University of Pennsylvania（宾夕法尼亚大学）
Motorsport has always been an enabler for technological advancement, and the same applies to the autonomous driving industry. The team TUM Auton-omous Motorsports will participate in the Indy Autonomous Challenge in Octo-ber 2021 to benchmark its self-driving software-stack by racing one out of ten autonomous Dallara AV-21 racecars at the Indianapolis Motor Speedway. The first part of this paper explains the reasons for entering an autonomous vehicle race from an academic perspective: It allows focusing on several edge cases en-countered by autonomous vehicles, such as challenging evasion mane…
Iterative learning control has been successfully used for several decades to improve the performance of control systems that perform a single repeated task. Using information from prior control executions, learning controllers gradually determine open-loop control inputs whose reference tracking performance can exceed that of traditional feedback-feedforward control algorithms. This paper considers iterative learning control for a previously unexplored field: autonomous racing. Racecars are driven multiple laps around the same sequence of turns while operating near the physical limits of tire-…
The challenges presented in an autonomous racing situation are distinct from those faced in regular autonomous driving and require faster end-to-end algorithms and consideration of a longer horizon in determining optimal current actions keeping in mind upcoming maneuvers and situations. In this paper, we propose an end-to-end method for autonomous racing that takes in as inputs video information from an onboard camera and determines final steering and throttle control actions. We use the following split to construct such a method (1) learning a low dimensional representation of the scene, (2) …
University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；RWTH Aachen University（亚琛工业大学）
In many robotic tasks, such as autonomous drone racing, the goal is to travel through a set of waypoints as fast as possible. A key challenge for this task is planning the time-optimal trajectory, which is typically solved by assuming perfect knowledge of the waypoints to pass in advance. The resulting solution is either highly specialized for a single-track layout, or suboptimal due to simplifying assumptions about the platform dynamics. In this work, a new approach to near-time-optimal trajectory generation for quadrotors is presented. Leveraging deep reinforcement learning and relative gate…
University of Zurich（苏黎世大学）；ETH Zurich（苏黎世联邦理工学院）；Intel Labs（英特尔实验室）
Autonomous agile flight brings up fundamental challenges in robotics, such as coping with unreliable state estimation, reacting optimally to dynamically changing environments, and coupling perception and action in real time under severe resource constraints. In this paper, we consider these challenges in the context of autonomous, vision-based drone racing in dynamic environments. Our approach combines a convolutional neural network (CNN) with a state-of-the-art path-planning and control system. The CNN directly maps raw images into a robust representation in the form of a waypoint and desired…
High-Fidelity Multi-Body Simulator for Autonomous Racing
University of Modena and Reggio Emilia（摩德纳与雷焦艾米利亚大学）；Megaride S.r.l.（Megaride 有限责任公司）；University of Naples Federico II（那不勒斯费德里科二世大学）；Claytex（Claytex 公司）
Longitudinal Control for Autonomous Racing with Combustion Engine Vehicles
Technical University of Munich, Germany（慕尼黑技术大学）；School of Engineering & Design（工程与设计学院）；Department of Engineering Physics and Computation（工程物理与计算系）；Institute of Automatic Control（自动控制研究所）；Department of Mobility Systems Engineering（移动系统工程系）；Institute of Automotive Technology（汽车技术研究所）
A2RL V\textsubscript{max}: The A2RL autonomous racing dataset for long-range, high-speed perception and multi-vehicle interaction
Institute of Measurement and Control Systems, Karlsruhe Institute of Technology（测量与控制系统研究所，卡尔斯鲁厄理工学院）；Institute of Automotive Technology, Technical University of Munich（汽车技术研究所，慕尼黑工业大学）；Professorship of Autonomous Vehicle Systems, Technical University of Munich（自动驾驶车辆系统教授职位，慕尼黑工业大学）；University of Modena and Reggio Emilia（摩德纳大学和雷焦艾米利亚大学）；Humda Lab, Széchenyi István University（胡姆达实验室，塞切尼·伊什特万大学）；Politecnico di Milano（米兰理工大学）；Constructor University（康斯坦丁大学）；Beijing Institute of Technology（北京理工大学）；Nanyang Technological University（南洋理工大学）；Code 19 Racing（代码19赛车）；Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich（慕尼黑机器人与机器智能研究所（MIRMI），慕尼黑工业大学）
