Safety-critical Autonomous Systems require trustworthy and transparent decision-making process to be deployable in the real world. The advancement of Machine Learning introduces high performance but largely through black-box algorithms. We focus the discussion of explainability specifically with Autonomous Vehicles (AVs). As a safety-critical system, AVs provide the unique opportunity to utilize cutting-edge Machine Learning techniques while requiring transparency in decision making. Interpretability in every action the AV takes becomes crucial in post-hoc analysis where blame assignment might be necessary. In this paper, we provide positioning on how researchers could consider incorporating explainability and interpretability into design and optimization of separate Autonomous Vehicle modules including Perception, Planning, and Control.
Technological advances in the automotive industry are bringing automated driving closer to road use. However, one of the most important factors affecting public acceptance of automated vehicles (AVs) is the public's trust in AVs. Many factors can influence people's trust, including perception of risks and benefits, feelings, and knowledge of AVs. This study aims to use these factors to predict people's dispositional and initial learned trust in AVs using a survey study conducted with 1175 participants. For each participant, 23 features were extracted from the survey questions to capture his or…
We present Pylot, a platform for autonomous vehicle (AV) research and development, built with the goal to allow researchers to study the effects of the latency and accuracy of their models and algorithms on the end-to-end driving behavior of an AV. This is achieved through a modular structure enabled by our high-performance dataflow system that represents AV software pipeline components (object detectors, motion planners, etc.) as a dataflow graph of operators which communicate on data streams using timestamped messages. Pylot readily interfaces with popular AV simulators like CARLA, and is ea…
FlightGoggles is a photorealistic sensor simulator for perception-driven robotic vehicles. The key contributions of FlightGoggles are twofold. First, FlightGoggles provides photorealistic exteroceptive sensor simulation using graphics assets generated with photogrammetry. Second, it provides the ability to combine (i) synthetic exteroceptive measurements generated in silico in real time and (ii) vehicle dynamics and proprioceptive measurements generated in motio by vehicle(s) in a motion-capture facility. FlightGoggles is capable of simulating a virtual-reality environment around autonomous ve…
Road accidents are estimated to be the ninth leading cause of death across all age groups globally. 1.25 million people die annually from road accidents and Africa has the highest rate of road fatalities [1]. Research shows that three out of five road accidents are caused by driver-related behavioral factors [2]. Self-driving technology has the potential of saving lives lost to these preventable road accidents. Africa accounts for the majority of road fatalities and as such would benefit immensely from this technology. However, financial constraints prevent viable experimentation and research …
Kettering University（凯特林大学）；University of Michigan-Dearborn（密歇根大学迪尔伯恩分校）
This paper illustrates the MIR (Mobile Intelligent Robotics) Vehicle: a feasible option of transforming an electric ride-on-car into a modular Graphics Processing Unit (GPU) powered autonomous platform equipped with the capability that supports test and deployment of various intelligent autonomous vehicles algorithms. To use a platform for research, two components must be provided: perception and control. The sensors such as incremental encoders, an Inertial Measurement Unit (IMU), a camera, and a LIght Detection And Ranging (LIDAR) must be able to be installed on the platform to add the capab…
Autonomous vehicle (AV) stacks are typically built in a modular fashion, with explicit components performing detection, tracking, prediction, planning, control, etc. While modularity improves reusability, interpretability, and generalizability, it also suffers from compounding errors, information bottlenecks, and integration challenges. To overcome these challenges, a prominent approach is to convert the AV stack into an end-to-end neural network and train it with data. While such approaches have achieved impressive results, they typically lack interpretability and reusability, and they eschew…
We were able to complete the full integration of the Robotic Technology Kernel (RTK) into an electric vehicle by-wire system using lidar and GPS sensors. The solution included a mobile application to interface with the RTK-enabled autonomous vehicle. Altogether the system was designed to be modular, using the concepts of message-based software design that is built into the Robot Operating System (ROS), which is at the foundation of RTK. The team worked incrementally to develop working software to demonstrate each milestone on the path to successfully completing the RTK integration for the deve…
Nanyang Technological University（南洋理工大学）；University of Oxford（牛津大学）
Integrating trajectory prediction to the decision-making and planning modules of modular autonomous driving systems is expected to improve the safety and efficiency of self-driving vehicles. However, a vehicle's future trajectory prediction is a challenging task since it is affected by the social interactive behaviors of neighboring vehicles, and the number of neighboring vehicles can vary in different situations. This work proposes a GNN-RNN based Encoder-Decoder network for interaction-aware trajectory prediction, where vehicles' dynamics features are extracted from their historical tracks u…
AV4EV: Open-Source Modular Autonomous Electric Vehicle Platform for Making Mobility Research Accessible
School of Engineering and Applied Science, University of Pennsylvania（宾夕法尼亚大学工程与应用科学学院）；Autoware Foundation Center of Excellence for Autonomous Driving（Autoware基金会自动驾驶卓越中心）；Professorship of Autonomous Vehicle Systems, Technical University of Munich（慕尼黑工业大学自动驾驶系统教授席位）；Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University in Prague（布拉格捷克技术大学捷克信息学、机器人学与控制论研究所）；Red Hat Research（红帽研究部）
FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院（KIT））
Behavior planning and decision-making are some of the biggest challenges for highly automated systems. A fully automated vehicle (AV) is confronted with numerous tactical and strategical choices. Most state-of-the-art AV platforms implement tactical and strategical behavior generation using finite state machines. However, these usually result in poor explainability, maintainability and scalability. Research in robotics has raised many architectures to mitigate these problems, most interestingly behavior-based systems and hybrid derivatives. Inspired by these approaches, we propose a hierarchic…
University of Oxford（牛津大学）；University of Nottingham（诺丁汉大学）
The automotive industry has witnessed an increasing level of development in the past decades; from manufacturing manually operated vehicles to manufacturing vehicles with a high level of automation. With the recent developments in Artificial Intelligence (AI), automotive companies now employ blackbox AI models to enable vehicles to perceive their environments and make driving decisions with little or no input from a human. With the hope to deploy autonomous vehicles (AV) on a commercial scale, the acceptance of AV by society becomes paramount and may largely depend on their degree of transpare…
Jabalpur Engineering College（贾巴尔普尔工程学院）；University of Cincinnati（辛辛那提大学）
... This paper is to describe exploratory research on the design of a modular autonomous mobile robot controller. The controller incorporates a fuzzy logic [8] [9] approach for steering and speed control [37], a FL approach for ultrasound sensing and an overall expert system for guidance. The advantages of a modular system are related to portability and transportability, i.e. any vehicle can become autonomous with minimal modifications. A mobile robot test bed has been constructed in university of Cincinnati using a golf cart base. This cart has full speed control with guidance provided by a v…
Technical University of Munich（慕尼黑工业大学）；Artisense；Audi Electronics Venture（奥迪电子创投）
Even though end-to-end supervised learning has shown promising results for sensorimotor control of self-driving cars, its performance is greatly affected by the weather conditions under which it was trained, showing poor generalization to unseen conditions. In this paper, we show how knowledge can be transferred using semantic maps to new weather conditions without the need to obtain new ground truth data. To this end, we propose to divide the task of vehicle control into two independent modules: a control module which is only trained on one weather condition for which labeled steering data is…
Iran University of Science and Technology（伊朗科技大学）；Amirkabir University of Technology（阿米卡比尔理工大学）；University of Calgary（卡尔加里大学）
A significant portion of driving hazards is caused by human error and disregard for local driving regulations; Consequently, an intelligent assistance system can be beneficial. This paper proposes a novel vision-based modular package to ensure drivers' safety by perceiving the environment. Each module is designed based on accuracy and inference time to deliver real-time performance. As a result, the proposed system can be implemented on a wide range of vehicles with minimum hardware requirements. Our modular package comprises four main sections: lane detection, object detection, segmentation, …
To maximize safety and driving comfort, autonomous driving systems can benefit from implementing foresighted action choices that take different potential scenario developments into account. While artificial scene prediction methods are making fast progress, an attentive human driver may still be able to identify relevant contextual features which are not adequately considered by the system or for which the human driver may have a lack of trust into the system's capabilities to treat them appropriately. We implement an approach that lets a human driver quickly and intuitively supplement scene p…
Traditional aerial vehicles are usually custom-designed for specific tasks. Although they offer an efficient solution, they are not always able to adapt to changes in the task specification, e.g., increasing the payload. This applies to quadrotors, having a maximum payload and only four controllable degrees of freedom, limiting their adaptability to the task's variations. We propose a versatile modular robotic system that can increase its payload and degrees of freedom by assembling heterogeneous modules; we call it H-ModQuad. It consists of cuboid modules propelled by quadrotors with tilted p…
SkyRover: A Modular Simulator for Cross-Domain Pathfinding
School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院）；School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；School of Mathematical Sciences, East China Normal University（华东师范大学数学科学学院）
University of Waterloo（滑铁卢大学）；University of Toronto（多伦多大学）
Inter-vehicle communication for autonomous vehicles (AVs) stands to provide significant benefits in terms of perception robustness. We propose a novel approach for AVs to communicate perceptual observations, tempered by trust modelling of peers providing reports. Based on the accuracy of reported object detections as verified locally, communicated messages can be fused to augment perception performance beyond line of sight and at great distance from the ego vehicle. Also presented is a new synthetic dataset which can be used to test cooperative perception. The TruPercept dataset includes unrel…
University of Pennsylvania（宾夕法尼亚大学）；Technical University of Munich（慕尼黑工业大学）；Lehigh University（理海大学）；University of Texas Austin（德克萨斯大学奥斯汀分校）；Poznan University of Technology（波兹南理工大学）；University of Virginia（弗吉尼亚大学）；Clemson University（克莱姆森大学）；Toyota Research Institute（丰田研究院）
Although robotics courses are well established in higher education, the courses often focus on theory and sometimes lack the systematic coverage of the techniques involved in developing, deploying, and applying software to real hardware. Additionally, most hardware platforms for robotics teaching are low-level toys aimed at younger students at middle-school levels. To address this gap, an autonomous vehicle hardware platform, called F1TENTH, is developed for teaching autonomous systems hands-on. This article describes the teaching modules and software stack for teaching at various educational …
Leading autonomous vehicle (AV) platforms and testing infrastructures are, unfortunately, proprietary and closed-source. Thus, it is difficult to evaluate how well safety-critical AVs perform and how safe they truly are. Similarly, few platforms exist for much-needed multi-agent analysis. To provide a starting point for analysis of sensor fusion and collaborative & distributed sensing, we design an accessible, modular sensing platform with AVstack. We build collaborative and distributed camera-radar fusion algorithms and demonstrate an evaluation ecosystem of AV datasets, physics-based simulat…
In this work, we propose a hierarchical reinforcement learning (HRL) structure which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals. In this hierarchical structure, the network is capable of 1) learning one task with multiple sub-goals simultaneously; 2) extracting attentions of states according to changing sub-goals during the learning process; 3) reusing the well-trained network of sub-goals for other similar tasks with the same sub-goals. The states are defined as processed observations which are transmitted from the perception s…
University of Oxford（牛津大学）；Umeå University（于默奥大学）；University of Nottingham（诺丁汉大学）；Oxford Robotics Institute（牛津机器人研究所）
In commentary driving, drivers verbalise their observations, assessments and intentions. By speaking out their thoughts, both learning and expert drivers are able to create a better understanding and awareness of their surroundings. In the intelligent vehicle context, automated driving commentary can provide intelligible explanations about driving actions, thereby assisting a driver or an end-user during driving operations in challenging and safety-critical scenarios. In this paper, we conducted a field study in which we deployed a research vehicle in an urban environment to obtain data. While…
Flow: A Modular Learning Framework for Mixed Autonomy Traffic
Laboratory for Information and Decision Systems, Massachusetts Institute of Technology（信息与决策实验室，麻省理工学院）；Institute of Data, Systems, and Society, Massachusetts Institute of Technology（数据、系统与社会研究所，麻省理工学院）；Department of Mechanical Engineering, University of California, Berkeley（机械工程系，加州大学伯克利分校）
The goal of this paper is to generate simulations with real-world collision scenarios for training and testing autonomous vehicles. We use numerous dashcam crash videos uploaded on the internet to extract valuable collision data and recreate the crash scenarios in a simulator. We tackle the problem of extracting 3D vehicle trajectories from videos recorded by an unknown and uncalibrated monocular camera source using a modular approach. A working architecture and demonstration videos along with the open-source implementation are provided with the paper.
Modular Adaptive Aerial Manipulation under Unknown Dynamic Coupling Forces
The University of Manchester（曼彻斯特大学）；Czech Technical University in Prague（布拉格捷克技术大学）；International Institute of Information Technology Hyderabad（海德拉巴国际信息技术学院）；Southeast University（东南大学）
Graduate School of Oceanography, University of Rhode Island（罗德岛大学海洋研究生院）；Department of Ocean Engineering, University of Rhode Island（罗德岛大学海洋工程系）；Department of Mechanical, Industrial, and Systems Engineering, University of Rhode Island（罗德岛大学机械、工业与系统工程系）；Department of Electrical and Computer Engineering, Stevens Institute of Technology（史蒂文斯理工学院电气与计算机工程系）；Department of Marine Technology, Norwegian University of Science and Technology（挪威科学技术大学海洋技术系）
