This paper presents the design of a tune-free (human-out-of-the-loop parameter tuning) control framework, aiming at accelerating large scale autonomous driving system deployed on various vehicles and driving environments. The framework consists of three machine-learning-based procedures, which jointly automate the control parameter tuning for autonomous driving, including: a learning-based dynamic modeling procedure, to enable the control-in-the-loop simulation with highly accurate vehicle dynamics for parameter tuning; a learning-based open-loop mapping procedure, to solve the feedforward control parameters tuning; and more significantly, a Bayesian-optimization-based closed-loop parameter tuning procedure, to automatically tune feedback control (PID, LQR, MRAC, MPC, etc.) parameters in simulation environment. The paper shows an improvement in control performance with a significant increase in parameter tuning efficiency, in both simulation and road tests. This framework has been validated on different vehicles in US and China.
Transformer-Based Autonomous Driving Models and Deployment-Oriented Compression: A Survey
Renmin University of China（中国人民大学）；Artificial Intelligence Innovation and Incubation Institute, Fudan University（复旦大学人工智能创新与孵化院）；Shanghai Academy of AI for Science（上海人工智能科学研究院）；Department of houmo.ai（houmo.ai部门）
Bench2Drive-Robust: Benchmarking Closed-Loop Autonomous Driving under Deployment Perturbations
Institute of Trustworthy Embodied AI (TEAI)（可信具身人工智能研究院）；Great Wall Motor（长城汽车）；Sch. of Computer Science & Sch. of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学计算机学院及人工智能学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）；University of Science and Technology of China（中国科学技术大学）
PerceptIn（感知创新公司）；South China University of Technology（华南理工大学）；Northwestern University（西北大学）
The infrastructure-vehicle cooperative autonomous driving approach depends on the cooperation between intelligent roads and intelligent vehicles. This approach is not only safer but also more economical compared to the traditional on-vehicle-only autonomous driving approach. In this paper, we introduce our real-world deployment experiences of cooperative autonomous driving, and delve into the details of new challenges and opportunities. Specifically, based on our progress towards commercial deployment, we follow a three-stage development roadmap of the cooperative autonomous driving approach:i…
Cross-cultural Deployment of Autonomous Vehicles Using Data-light Inverse Reinforcement Learning
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Beijing Institute of Technology（北京理工大学）
Volkswagen AG（大众汽车公司）；Technische Universität Darmstadt（达姆施塔特工业大学）
Behavior and motion planning play an important role in automated driving. Traditionally, behavior planners instruct local motion planners with predefined behaviors. Due to the high scene complexity in urban environments, unpredictable situations may occur in which behavior planners fail to match predefined behavior templates. Recently, general-purpose planners have been introduced, combining behavior and local motion planning. These general-purpose planners allow behavior-aware motion planning given a single reward function. However, two challenges arise: First, this function has to map a comp…
The Hong Kong University of Science and Technology（香港科技大学）
End-to-end visual-based imitation learning has been widely applied in autonomous driving. When deploying the trained visual-based driving policy, a deterministic command is usually directly applied without considering the uncertainty of the input data. Such kind of policies may bring dramatical damage when applied in the real world. In this paper, we follow the recent real-to-sim pipeline by translating the testing world image back to the training domain when using the trained policy. In the translating process, a stochastic generator is used to generate various images stylized under the train…
ENSTA ParisTech（巴黎高等科技学院）；AKKA Technologies（阿卡科技公司）；Valeo Egypt（法雷奥埃及公司）；Valeo Vision Systems（法雷奥视觉系统公司）；Galway-Mayo Institute of Technology（戈尔韦-梅奥理工学院）
Deep Reinforcement Learning (DRL) has become increasingly powerful in recent years, with notable achievements such as Deepmind's AlphaGo. It has been successfully deployed in commercial vehicles like Mobileye's path planning system. However, a vast majority of work on DRL is focused on toy examples in controlled synthetic car simulator environments such as TORCS and CARLA. In general, DRL is still at its infancy in terms of usability in real-world applications. Our goal in this paper is to encourage real-world deployment of DRL in various autonomous driving (AD) applications. We first provide …
Explainable deep learning improves human mental models of self-driving cars
Computer Science & Artificial Intelligence Laboratory (CSAIL), Massachusetts Institute of Technology（计算机科学与人工智能实验室（CSAIL），麻省理工学院）；Motional AD Inc.（Motional AD公司）；Department of Psychology and Center for Brain Science, Harvard University（心理学系和大脑科学中心，哈佛大学）；Department of Aeronautics and Astronautics, Massachusetts Institute of Technology（航空与宇航系，麻省理工学院）
Behavioral Safety Assessment towards Large-scale Deployment of Autonomous Vehicles, Part I: Methodology
University of Michigan Transportation Research Institute（密歇根大学交通研究所）；University of Michigan（密歇根大学）；The University of Hong Kong（香港大学）；Laplace Intelligence（拉普拉斯智能）；Tsinghua University（清华大学）
School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与航天工程学院）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Autonomous driving clouds provide essential services to support autonomous vehicles. Today these services include but not limited to distributed simulation tests for new algorithm deployment, offline deep learning model training, and High-Definition (HD) map generation. These services require infrastructure support including distributed computing, distributed storage, as well as heterogeneous computing. In this paper, we present the details of how we implement a unified autonomous driving cloud infrastructure, and how we support these services on top of this infrastructure.
Experimental Validation of Safe MPC for Autonomous Driving in Uncertain Environments
Chalmers University of Technology（查尔姆斯理工大学）；Zenseact AB（Zenseact公司）；IMT School for Advanced Studies Lucca（卢卡IMT高等研究院）；Università di Modena e Reggio Emilia（摩德纳-雷焦艾米利亚大学）
Toyota Research Institute（丰田研究院）；Georgia Institute of Technology（佐治亚理工学院）
Personalization of autonomous vehicles (AV) may significantly increase trust, use, and acceptance. In particular, we hypothesize that the similarity of an AV's driving style compared to the end-user's driving style will have a major impact on end-user's willingness to use the AV. To investigate the impact of driving style on user acceptance, we 1) develop a data-driven approach to personalize driving style and 2) demonstrate that personalization significantly impacts attitudes towards AVs. Our approach learns a high-level model that tunes low-level controllers to ensure safe and personalized c…
UniSTPA: A Safety Analysis Framework for End-to-End Autonomous Driving
National Key Laboratory of Automotive Chassis Integration and Bionics, Jilin University, Changchun, China（吉林大学汽车底盘集成与仿生国家级重点实验室）；School of Engineering and Physical Sciences, Heriot‐Watt University, Edinburgh, United Kingdom（赫瑞-沃森大学工程与物理科学学院）
University of Oxford（牛津大学）；Aalborg University（奥尔堡大学）
This paper discusses ongoing work in demonstrating research in mobile autonomy in challenging driving scenarios. In our approach, we address fundamental technical issues to overcome critical barriers to assurance and regulation for large-scale deployments of autonomous systems. To this end, we present how we build robots that (1) can robustly sense and interpret their environment using traditional as well as unconventional sensors; (2) can assess their own capabilities; and (3), vitally in the purpose of assurance and trust, can provide causal explanations of their interpretations and assessme…
The majority of current approaches in autonomous driving rely on High-Definition (HD) maps which detail the road geometry and surrounding area. Yet, this reliance is one of the obstacles to mass deployment of autonomous vehicles due to poor scalability of such prior maps. In this paper, we tackle the problem of online road map extraction via leveraging the sensory system aboard the vehicle itself. To this end, we design a structured model where a graph representation of the road network is generated in a hierarchical fashion within a fully convolutional network. The method is able to handle co…
Navya（纳维亚）；Valeo（法雷奥）；ENSTA Paris, Institut Polytechnique de Paris（巴黎理工学院巴黎高等先进技术学院）；AKKA Technologies（AKKA科技公司）；National University of Ireland, Galway（爱尔兰国立高威大学）
With the development of deep representation learning, the domain of reinforcement learning (RL) has become a powerful learning framework now capable of learning complex policies in high dimensional environments. This review summarises deep reinforcement learning (DRL) algorithms and provides a taxonomy of automated driving tasks where (D)RL methods have been employed, while addressing key computational challenges in real world deployment of autonomous driving agents. It also delineates adjacent domains such as behavior cloning, imitation learning, inverse reinforcement learning that are relate…
University of Pennsylvania（宾夕法尼亚大学）；Honda Research Institute（本田研究院）
We view intersection handling on autonomous vehicles as a reinforcement learning problem, and study its behavior in a transfer learning setting. We show that a network trained on one type of intersection generally is not able to generalize to other intersections. However, a network that is pre-trained on one intersection and fine-tuned on another performs better on the new task compared to training in isolation. This network also retains knowledge of the prior task, even though some forgetting occurs. Finally, we show that the benefits of fine-tuning hold when transferring simulated intersecti…
One of the challenges in vision-based driving trajectory generation is dealing with out-of-distribution scenarios. In this paper, we propose a domain generalization method for vision-based driving trajectory generation for autonomous vehicles in urban environments, which can be seen as a solution to extend the Invariant Risk Minimization (IRM) method in complex problems. We leverage an adversarial learning approach to train a trajectory generator as the decoder. Based on the pre-trained decoder, we infer the latent variables corresponding to the trajectories, and pre-train the encoder by regre…
Nav-EE: Navigation-Guided Early Exiting for Efficient Vision-Language Models in Autonomous Driving
Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；Department of Computer Science, McGill University（麦吉尔大学计算机科学系）；Department of Computer Science, Mohamed bin Zayed University of Artificial Intelligence（马尔代夫穆罕默德· bin·扎耶德人工智能大学计算机科学系）
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Delaware（特拉华大学）；University of Chinese Academy of Sciences（中国科学院大学）
Infrastructure-Vehicle Cooperative Autonomous Driving (IVCAD) is a new paradigm of autonomous driving, which relies on the cooperation between intelligent roads and autonomous vehicles. This paradigm has been shown to be safer and more efficient compared to the on-vehicle-only autonomous driving paradigm. Our real-world deployment data indicates that the effectiveness of IVCAD is constrained by reliability and performance of commercial communication networks. This paper targets this exact problem, and proposes INTERNEURON, a middleware to achieve high communication reliability between intellig…
Toward a Full-Stack Co-Simulation Platform for Testing of Automated Driving Systems
Institute of Automotive Engineering（汽车工程研究所）；Graz University of Technology（格拉茨理工大学）；College of Transportation Engineering（交通工程学院）；Tongji University（同济大学）
LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trends
Purdue University（普渡大学）；Institute for Physical Artificial Intelligence (IPAI), Purdue University（普渡大学物理人工智能研究所）；Department of Computer Science, Purdue University（普渡大学计算机科学系）；InfoTech Labs, Toyota Motor North America（丰田北美信息技术实验室）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The behavior of self driving cars may differ from people expectations, (e.g. an autopilot may unexpectedly relinquish control). This expectation mismatch can cause potential and existing users to distrust self driving technology and can increase the likelihood of accidents. We propose a simple but effective framework, AutoPreview, to enable consumers to preview a target autopilot potential actions in the real world driving context before deployment. For a given target autopilot, we design a delegate policy that replicates the target autopilot behavior with explainable action representations, w…
Autonomous Driving vehicles (ADV) are on road with large scales. For safe and efficient operations, ADVs must be able to predict the future states and iterative with road entities in complex, real-world driving scenarios. How to migrate a well-trained prediction model from one geo-fenced area to another is essential in scaling the ADV operation and is difficult most of the time since the terrains, traffic rules, entities distributions, driving/walking patterns would be largely different in different geo-fenced operation areas. In this paper, we introduce a highly automated learning-based predi…
Autonomous driving is getting a lot of attention in the last decade and will be the hot topic at least until the first successful certification of a car with Level 5 autonomy. There are many public datasets in the academic community. However, they are far away from what a robust industrial production system needs. There is a large gap between academic and industrial setting and a substantial way from a research prototype, built on public datasets, to a deployable solution which is a challenging task. In this paper, we focus on bad practices that often happen in the autonomous driving from an i…
Wayne State University（韦恩州立大学）；Anhui University（安徽大学）
The recent proliferation of computing technologies (e.g., sensors, computer vision, machine learning, and hardware acceleration), and the broad deployment of communication mechanisms (e.g., DSRC, C-V2X, 5G) have pushed the horizon of autonomous driving, which automates the decision and control of vehicles by leveraging the perception results based on multiple sensors. The key to the success of these autonomous systems is making a reliable decision in real-time fashion. However, accidents and fatalities caused by early deployed autonomous vehicles arise from time to time. The real traffic envir…
PerceptIn（PerceptIn公司）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）；South China University of Technology（华南理工大学）；University of Sydney（悉尼大学）
This paper is the first to provide a thorough system design overview along with the fusion methods selection criteria of a real-world cooperative autonomous driving system, named Infrastructure-Augmented Autonomous Driving or IAAD. We present an in-depth introduction of the IAAD hardware and software on both road-side and vehicle-side computing and communication platforms. We extensively characterize the IAAD system in the context of real-world deployment scenarios and observe that the network condition that fluctuates along the road is currently the main technical roadblock for cooperative au…
SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Chongqing Changan Technology Co., Ltd.（重庆长安科技有限公司）；Civil Aviation University of China（中国民航大学）；Beihang University（北京航空航天大学）；Guilin University of Electronic Technology（桂林电子科技大学）
