Decision Making for Autonomous Driving via Augmented Adversarial Inverse Reinforcement Learning
University of California, Berkeley（加州大学伯克利分校）；Zenseact（森赛克特）；Chalmers University of Technology（查尔姆斯理工大学）；Peking University（北京大学）；Google Research（谷歌研究院）
Making decisions in complex driving environments is a challenging task for autonomous agents. Imitation learning methods have great potentials for achieving such a goal. Adversarial Inverse Reinforcement Learning (AIRL) is one of the state-of-art imitation learning methods that can learn both a behavioral policy and a reward function simultaneously, yet it is only demonstrated in simple and static environments where no interactions are introduced. In this paper, we improve and stabilize AIRL's performance by augmenting it with semantic rewards in the learning framework. Additionally, we adapt the augmented AIRL to a more practical and challenging decision-making task in a highly interactive environment in autonomous driving. The proposed method is compared with four baselines and evaluated by four performance metrics. Simulation results show that the augmented AIRL outperforms all the baseline methods, and its performance is comparable with that of the experts on all of the four metrics.
Robust Driving Control for Autonomous Vehicles: An Intelligent General-sum Constrained Adversarial Reinforcement Learning Approach
Beijing Key Laboratory of Security and Privacy in Intelligent Transportation（北京智能交通安全与隐私重点实验室）；Beijing Jiaotong University（北京交通大学）；College of Computing and Data Science（计算与数据科学学院）；Nanyang Technological University（南洋理工大学）；School of Computer Science and Technology（计算机科学与技术学院）；Taiyuan University of Technology（太原科技大学）；School of Electronics and Information Engineering（电子与信息工程学院）
Autonomous cars are well known for being vulnerable to adversarial attacks that can compromise the safety of the car and pose danger to other road users. To effectively defend against adversaries, it is required to not only test autonomous cars for finding driving errors but to improve the robustness of the cars to these errors. To this end, in this paper, we propose a two-step methodology for autonomous cars that consists of (i) finding failure states in autonomous cars by training the adversarial driving agent, and (ii) improving the robustness of autonomous cars by retraining them with effe…
We examine the problem of adversarial reinforcement learning for multi-agent domains including a rule-based agent. Rule-based algorithms are required in safety-critical applications for them to work properly in a wide range of situations. Hence, every effort is made to find failure scenarios during the development phase. However, as the software becomes complicated, finding failure cases becomes difficult. Especially in multi-agent domains, such as autonomous driving environments, it is much harder to find useful failure scenarios that help us improve the algorithm. We propose a method for eff…
An open problem in autonomous vehicle safety validation is building reliable models of human driving behavior in simulation. This work presents an approach to learn neural driving policies from real world driving demonstration data. We model human driving as a sequential decision making problem that is characterized by non-linearity and stochasticity, and unknown underlying cost functions. Imitation learning is an approach for generating intelligent behavior when the cost function is unknown or difficult to specify. Building upon work in inverse reinforcement learning (IRL), Generative Adversa…
DRARL: Disengagement-Reason-Augmented Reinforcement Learning for Efficient Improvement of Autonomous Driving Policy
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动学院）；Didi Global（滴滴出行）；Department of Civil and Environmental Engineering, University of Michigan（密歇根大学土木与环境工程系）；Lab for High Technology, Tsinghua University（清华大学高科技实验室）；Institute for AI Industry Research, Tsinghua University（清华大学人工智能产业研究院）
University of Southern California（南加州大学）；Toyota Research Institute（丰田研究院）
Autonomous systems such as self-driving cars and general-purpose robots are safety-critical systems that operate in highly uncertain and dynamic environments. We propose an interactive multi-agent framework where the system-under-design is modeled as an ego agent and its environment is modeled by a number of adversarial (ado) agents. For example, a self-driving car is an ego agent whose behavior is influenced by ado agents such as pedestrians, bicyclists, traffic lights, road geometry etc. Given a logical specification of the correct behavior of the ego agent, and a set of constraints that enc…
Autonomous driving is a complex task, which has been tackled since the first self-driving car ALVINN in 1989, with a supervised learning approach, or behavioral cloning (BC). In BC, a neural network is trained with state-action pairs that constitute the training set made by an expert, i.e., a human driver. However, this type of imitation learning does not take into account the temporal dependencies that might exist between actions taken in different moments of a navigation trajectory. These type of tasks are better handled by reinforcement learning (RL) algorithms, which need to define a rewar…
Montreal Institute for Learning Algorithms, Université de Montréal（蒙特利尔学习算法研究所，蒙特利尔大学）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；IVADO
Despite the successes in capturing continuous distributions, the application of generative adversarial networks (GANs) to discrete settings, like natural language tasks, is rather restricted. The fundamental reason is the difficulty of back-propagation through discrete random variables combined with the inherent instability of the GAN training objective. To address these problems, we propose Maximum-Likelihood Augmented Discrete Generative Adversarial Networks. Instead of directly optimizing the GAN objective, we derive a novel and low-variance objective using the discriminator's output that f…
With the evolution of various advanced driver assistance system (ADAS) platforms, the design of autonomous driving system is becoming more complex and safety-critical. The autonomous driving system simultaneously activates multiple ADAS functions; and thus it is essential to coordinate various ADAS functions. This paper proposes a randomized adversarial imitation learning (RAIL) method that imitates the coordination of autonomous vehicle equipped with advanced sensors. The RAIL policies are trained through derivative-free optimization for the decision maker that coordinates the proper ADAS fun…
Korea University（高丽大学）；Mofl Inc.（Mofl 公司）；Hallym University（翰林大学）；California State University, Long Beach（加州州立大学长滩分校）
Self-driving cars and autonomous driving research has been receiving considerable attention as major promising prospects in modern artificial intelligence applications. According to the evolution of advanced driver assistance system (ADAS), the design of self-driving vehicle and autonomous driving systems becomes complicated and safety-critical. In general, the intelligent system simultaneously and efficiently activates ADAS functions. Therefore, it is essential to consider reliable ADAS function coordination to control the driving system, safely. In order to deal with this issue, this paper p…
Valeo DVS Prague（法雷奥DVS布拉格）；Valeo CDV AI Cairo（法雷奥CDV人工智能开罗）；Valeo Vision Systems（法雷奥视觉系统）
Generative Adversarial Networks (GAN) have gained a lot of popularity from their introduction in 2014 till present. Research on GAN is rapidly growing and there are many variants of the original GAN focusing on various aspects of deep learning. GAN are perceived as the most impactful direction of machine learning in the last decade. This paper focuses on the application of GAN in autonomous driving including topics such as advanced data augmentation, loss function learning, semi-supervised learning, etc. We formalize and review key applications of adversarial techniques and discuss challenges …
ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving
The Hong Kong Polytechnic University, Hong Kong SAR, China（香港理工大学）；Tongji University, Shanghai, China（同济大学）；McGill University, Montreal, QC, Canada（麦吉尔大学）
There is considerable evidence that deep neural networks are vulnerable to adversarial perturbations applied directly to their digital inputs. However, it remains an open question whether this translates to vulnerabilities in real systems. For example, an attack on self-driving cars would in practice entail modifying the driving environment, which then impacts the video inputs to the car's controller, thereby indirectly leading to incorrect driving decisions. Such attacks require accounting for system dynamics and tracking viewpoint changes. We propose a scalable approach for finding adversari…
Stanford University（斯坦福大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
Recent developments in multi-agent imitation learning have shown promising results for modeling the behavior of human drivers. However, it is challenging to capture emergent traffic behaviors that are observed in real-world datasets. Such behaviors arise due to the many local interactions between agents that are not commonly accounted for in imitation learning. This paper proposes Reward Augmented Imitation Learning (RAIL), which integrates reward augmentation into the multi-agent imitation learning framework and allows the designer to specify prior knowledge in a principled fashion. We prove …
Evaluating the robustness of automated driving planners is a critical and challenging task. Although methodologies to evaluate vehicles are well established, they do not yet account for a reality in which vehicles with autonomous components share the road with adversarial agents. Our approach, based on probabilistic trust models, aims to help researchers assess the robustness of protections for machine learning-enabled planners against adversarial influence. In contrast with established practices that evaluate safety using the same evaluation dataset for all vehicles, we argue that adversarial…
LD-Scene: LLM-Guided Diffusion for Controllable Generation of Adversarial Safety-Critical Driving Scenarios
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）
Attack End-to-End Autonomous Driving through Module-Wise Noise
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；State Key Lab of Software Development Environment, Beihang University（北京航空航天大学软件开发环境国家重点实验室）；Shen Yuan Honors College, Beihang University（北京航空航天大学沈元荣誉学院）；School of Software, Beihang University（北京航空航天大学软件学院）
Indian Institute of Technology Kharagpur（印度理工学院卡拉格普尔分校）
Autonomous driving is one of the most recent topics of interest which is aimed at replicating human driving behavior keeping in mind the safety issues. We approach the problem of learning synthetic driving using generative neural networks. The main idea is to make a controller trainer network using images plus key press data to mimic human learning. We used the architecture of a stable GAN to make predictions between driving scenes using key presses. We train our model on one video game (Road Rash) and tested the accuracy and compared it by running the model on other maps in Road Rash to deter…
Enhancing Noise Robustness of Retrieval-Augmented Language Models with Adaptive Adversarial Training
University of Science and Technology of China（中国科学技术大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Shenzhen University（深圳大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
Vanderbilt University（范德堡大学）；Technical University of Munich（慕尼黑工业大学）
Despite recent advances in autonomous driving systems, accidents such as the fatal Uber crash in 2018 show these systems are still susceptible to edge cases. Such systems must be thoroughly tested and validated before being deployed in the real world to avoid such events. Testing in open-world scenarios can be difficult, time-consuming, and expensive. These challenges can be addressed by using driving simulators such as CARLA instead. A key part of such tests is adversarial testing, in which the goal is to find scenarios that lead to failures of the given system. While several independent effo…
VLADriver-RAG: Retrieval-Augmented Vision-Language-Action Models for Autonomous Driving
College of Automotive Engineering（汽车工程学院）；The National Key Laboratory of Automotive Chassis Integration and Bionics（汽车底盘集成与生物力学国家级重点实验室）；ReeFocus AI Technology（ReeFocus人工智能技术）
Case-based Reasoning Augmented Large Language Model Framework for Decision Making in Realistic Safety-Critical Driving Scenarios
Big Data Integration Research Center, National Institute of Information and Communications Technology (NICT), Tokyo, Japan（大数据整合研究室，日本信息通信技术研究所（NICT），东京）
Black-Box Adversarial Attack on Vision Language Models for Autonomous Driving
Beihang University（北京航空航天大学）；National University of Singapore（新加坡国立大学）；Aviation Industry Development Research Center of China（中国航空工业发展研究中心）；Nanyang Technological University（南洋理工大学）
Visual-Invariance-Augmented Feature Optimal Alignment for Transferable Adversarial Attacks against Closed-Source MLLMs
Nanyang Technological University（南洋理工大学）；Northeastern University（东北大学）；Chongqing University（重庆大学）；University of Chinese Academy of Sciences（中国科学院大学）；Sun Yat-sen University（中山大学）
