The commonly used metrics for motion prediction do not correlate well with a self-driving vehicle's system-level performance. The most common metrics are average displacement error (ADE) and final displacement error (FDE), which omit many features, making them poor self-driving performance indicators. Since high-fidelity simulations and track testing can be resource-intensive, the use of prediction metrics better correlated with full-system behavior allows for swifter iteration cycles. In this paper, we offer a conceptual framework for prediction evaluation highly specific to self-driving. We propose two complementary metrics that quantify the effects of motion prediction on safety (related to recall) and comfort (related to precision). Using a simulator, we demonstrate that our safety metric has a significantly better signal-to-noise ratio than displacement error in identifying unsafe events.
Despite recent advances in reinforcement learning (RL), its application in safety critical domains like autonomous vehicles is still challenging. Although punishing RL agents for risky situations can help to learn safe policies, it may also lead to highly conservative behavior. In this paper, we propose a distributional RL framework in order to learn adaptive policies that can tune their level of conservativity at run-time based on the desired comfort and utility. Using a proactive safety verification approach, the proposed framework can guarantee that actions generated from RL are fail-safe a…
Saarland University（萨尔大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
We present a novel hybrid learning method, HyLEAR, for solving the collision-free navigation problem for self-driving cars in POMDPs. HyLEAR leverages interposed learning to embed knowledge of a hybrid planner into a deep reinforcement learner to faster determine safe and comfortable driving policies. In particular, the hybrid planner combines pedestrian path prediction and risk-aware path planning with driving-behavior rule-based reasoning such that the driving policies also take into account, whenever possible, the ride comfort and a given set of driving-behavior rules. Our experimental perf…
Department of Computer Science, University of Memphis（孟菲斯大学计算机科学系）；University of Memphis（孟菲斯大学）
We present a novel adaptive cruise control (ACC) system namely SAINT-ACC: {S}afety-{A}ware {Int}elligent {ACC} system (SAINT-ACC) that is designed to achieve simultaneous optimization of traffic efficiency, driving safety, and driving comfort through dynamic adaptation of the inter-vehicle gap based on deep reinforcement learning (RL). A novel dual RL agent-based approach is developed to seek and adapt the optimal balance between traffic efficiency and driving safety/comfort by effectively controlling the driving safety model parameters and inter-vehicle gap based on macroscopic and microscopi…
Horizon Robotics；University of Hong Kong（香港大学）；University of the Chinese Academy of Sciences（中国科学院大学）；Nanjing University（南京大学）；Beijing Jiaotong University（北京交通大学）
The processing requirement of autonomous vehicles (AVs) for high-accuracy perception in complex scenarios can exceed the resources offered by the in-vehicle computer, degrading safety and comfort. This paper proposes a sensor frame processing rate (FPR) estimation model, Zhuyi, that quantifies the minimum safe FPR continuously in a driving scenario. Zhuyi can be employed post-deployment as an online safety check and to prioritize work. Experiments conducted using a multi-camera state-of-the-art industry AV system show that Zhuyi's estimated FPRs are conservative, yet the system can maintain sa…
While artificial-intelligence-based methods suffer from lack of transparency, rule-based methods dominate in safety-critical systems. Yet, the latter cannot compete with the first ones in robustness to multiple requirements, for instance, simultaneously addressing safety, comfort, and efficiency. Hence, to benefit from both methods they must be joined in a single system. This paper proposes a decision making and control framework, which profits from advantages of both the rule- and machine-learning-based techniques while compensating for their disadvantages. The proposed method embodies two co…
Tongji University（同济大学）；University of Washington（华盛顿大学）
A model used for velocity control during car following was proposed based on deep reinforcement learning (RL). To fulfil the multi-objectives of car following, a reward function reflecting driving safety, efficiency, and comfort was constructed. With the reward function, the RL agent learns to control vehicle speed in a fashion that maximizes cumulative rewards, through trials and errors in the simulation environment. A total of 1,341 car-following events extracted from the Next Generation Simulation (NGSIM) dataset were used to train the model. Car-following behavior produced by the model wer…
Motion Comfort Optimization for Autonomous Vehicles: Concepts, Methods, and Techniques
University of North Texas（北德克萨斯大学）；Fordham University（福特汉姆大学）；Qatar University（卡塔尔大学）；University of Nebraska at Kearney（内布拉斯加大学卡尼尔分校）；Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·扎耶德人工智能大学（MBZUAI））；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
Predictive Maneuver Planning with Deep Reinforcement Learning (PMP-DRL) for comfortable and safe autonomous driving
Robert Bosch Centre for Cyber-Physical Systems, Indian Institute of Science（印度科学学院罗伯特·博世信息物理系统中心）；Indian Institute of Science（印度科学学院）；PES university（PES大学）
Modern driver assistance systems as well as autonomous vehicles take their decisions based on local maps of the environment. These maps include, for example, surrounding moving objects perceived by sensors as well as routes and navigation information. Current research in the field of environment mapping is concerned with two major challenges. The first one is the integration of information from different sources e.g. on-board sensors like radar, camera, ultrasound and lidar, offline map data or backend information. The second challenge comprises in finding an abstract representation of this ag…
This review paper focuses on different aspects of perceived safety for a number of autonomous physical systems. This is a major aspect of robotics research, as more and more applications allow human and autonomous systems to share their space, with crucial implications both on safety and on its perception. The alternative terms used to express related concepts (e.g., psychological safety, trust, comfort, stress, fear, and anxiety) are listed and explained. Then, the available methods to assess perceived safety (i.e., questionnaires, physiological measurements, behavioral assessment, and direct…
It is envisioned that, in the future, most vehicles on our roadway will be controlled autonomously and will be connected via vehicle to everything (V2X) wireless communication networks. Developing a connected and automated vehicle (CAV) longitudinal controller, which will consider safety, comfort and operational efficiency simultaneously, is a challenge. A CAV longitudinal controller is a complex system where a vehicle senses immediate upstream vehicles using its sensors and receives information about its surroundings via wireless connectivity, and move forward accordingly. In this study, we d…
Reinforcement learning (RL) has recently been used for solving challenging decision-making problems in the context of automated driving. However, one of the main drawbacks of the presented RL-based policies is the lack of safety guarantees, since they strive to reduce the expected number of collisions but still tolerate them. In this paper, we propose an efficient RL-based decision-making pipeline for safe and cooperative automated driving in merging scenarios. The RL agent is able to predict the current situation and provide high-level decisions, specifying the operation mode of the low level…
Woven Planet United Kingdom Limited（Woven Planet英国有限公司）；Woven Planet North America, Inc.（Woven Planet北美公司）
The goal of autonomous vehicles is to navigate public roads safely and comfortably. To enforce safety, traditional planning approaches rely on handcrafted rules to generate trajectories. Machine learning-based systems, on the other hand, scale with data and are able to learn more complex behaviors. However, they often ignore that agents and self-driving vehicle trajectory distributions can be leveraged to improve safety. In this paper, we propose modeling a distribution over multiple future trajectories for both the self-driving vehicle and other road agents, using a unified neural network arc…
Shanghai Jiao Tong University（上海交通大学）；University of California, Berkeley（加州大学伯克利分校）
To generate safe and real-time trajectories for an autonomous vehicle in dynamic environments, path and speed decoupled planning methods are often considered. This paper studies speed planning, which mainly deals with dynamic obstacle avoidance given the planning path. The main challenges lie in the decisions in non-convex space and the trade-off between safety, comfort and efficiency performances. This work uses dynamic programming to search heuristic waypoints on the S-T graph and to construct convex feasible spaces. Further, a piecewise Bezier polynomials optimization approach with trapezoi…
A Communication-Latency-Aware Co-Simulation Platform for Safety and Comfort Evaluation of Cloud-Controlled ICVs
School of Automotive Studies, Tongji University（同济大学汽车学院）；Institute of Automotive Engineering, Graz University of Technology（技术大学格拉茨汽车工程研究所）；Department of Automotive Technologies, Budapest University of Technology and Economics（布达佩斯技术与经济大学汽车技术系）；Magyar Telekom Nyrt.（匈牙利马格yar Telekom公司）；Key Laboratory of Road and Traffic Engineering of the Ministry of Education, Tongji University（教育部道路与交通工程重点实验室，同济大学）
Navigating safely in urban environments remains a challenging problem for autonomous vehicles. Occlusion and limited sensor range can pose significant challenges to safely navigate among pedestrians and other vehicles in the environment. Enabling vehicles to quantify the risk posed by unseen regions allows them to anticipate future possibilities, resulting in increased safety and ride comfort. This paper proposes an algorithm that takes advantage of the known road layouts to forecast, quantify, and aggregate risk associated with occlusions and limited sensor range. This allows us to make predi…
Evaluation and Control Model Design of Human Factors for Autonomous Driving Systems
The Hong Kong University of Science and Technology（香港科技大学）；Shanghai Jiao Tong University（上海交通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
FiveAI Ltd（FiveAI有限公司）；University of Oxford（牛津大学）；University of Edinburgh（爱丁堡大学）
Recent road trials have shown that guaranteeing the safety of driving decisions is essential for the wider adoption of autonomous vehicle technology. One promising direction is to pose safety requirements as planning constraints in nonlinear, non-convex optimization problems of motion synthesis. However, many implementations of this approach are limited by uncertain convergence and local optimality of the solutions achieved, affecting overall robustness. To improve upon these issues, we propose a novel two-stage optimization framework: in the first stage, we find a solution to a Mixed-Integer …
To maximize safety and driving comfort, autonomous driving systems can benefit from implementing foresighted action choices that take different potential scenario developments into account. While artificial scene prediction methods are making fast progress, an attentive human driver may still be able to identify relevant contextual features which are not adequately considered by the system or for which the human driver may have a lack of trust into the system's capabilities to treat them appropriately. We implement an approach that lets a human driver quickly and intuitively supplement scene p…
NXP Semiconductors（恩智浦半导体）；Eindhoven University of Technology（埃因霍温理工大学）
Modern Automated Driving (AD) systems rely on safety measures to handle faults and to bring vehicle to a safe state. To eradicate lethal road accidents, car manufacturers are constantly introducing new perception as well as control systems. Contemporary automotive design and safety engineering best practices are suitable for analyzing system components in isolation, whereas today's highly complex and interdependent AD systems require novel approach to ensure resilience to multi-point failures. We present a holistic safety concept unifying advanced safety measures for handling multiple-point fa…
Integrating Decision-Making Into Differentiable Optimization Guided Learning for End-to-End Planning of Autonomous Vehicles
The Hong Kong University of Science and Technology（香港科技大学）；Lotus Robotics（路特斯机器人公司）；School of Mechanical and Aerospace Engineering, Nanyang Technological University（南洋理工大学机械与宇航工程学院）
本文提出将决策与轨迹规划建模为可微非线性优化并融合神经网络预测器，构建端到端自动驾驶规划框架，在 Waymo Open Motion 上验证其安全、效率与舒适性优势。
Rhodes University（罗德斯大学）；University of Ngaoundéré（恩冈代雷大学）
In South Africa, it is a common practice for people to leave their vehicles beside the road when traveling long distances for a short comfort break. This practice might increase human encounters with wildlife, threatening their security and safety. Here we intend to create awareness about wildlife fencing, using drone technology and computer vision algorithms to recognize and detect wildlife fences and associated features. We collected data at Amakhala and Lalibela private game reserves in the Eastern Cape, South Africa. We used wildlife electric fence data containing single and double fences …
Safe and Nonconservative Contingency Planning for Autonomous Vehicles via Online Learning-Based Reachable Set Barriers
The Hong Kong University of Science and Technology（香港科学与技术大学）；National University of Singapore（新加坡国立大学）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Current advances in the development of autonomous cars suggest that driverless cars may see wide-scale deployment in the near future. Research by both industry and academia is driven by potential benefits of this new technology, including reductions in fatalities and improvements in traffic and fuel efficiency as well as greater mobility for people who will or cannot drive cars themselves. A deciding factor for the adoption of self-driving cars besides safety will be the comfort of the passengers. This report looks at cost functions currently used in motion planning methods for autonomous on-r…
Follow-Bench: A Unified Motion Planning Benchmark for Socially-Aware Robot Person Following
Shenzhen Key Laboratory of Robotics and Computer Vision（深圳机器人与计算机视觉重点实验室）；Southern University of Science and Technology（南方科技大学）；Human-Robot Interfaces and Interaction Laboratory（人机交互实验室）；Istituto Italiano Di Tecnologia（意大利技术研究院）；Swiss Federal Technology Institute of Lausanne（洛桑联邦理工学院）
Beijing Institute of Technology（北京理工大学）；California PATH, University of California, Berkeley（加州大学伯克利分校加州PATH项目）
We apply Deep Q-network (DQN) with the consideration of safety during the task for deciding whether to conduct the maneuver. Furthermore, we design two similar Deep Q learning frameworks with quadratic approximator for deciding how to select a comfortable gap and just follow the preceding vehicle. Finally, a polynomial lane change trajectory is generated and Pure Pursuit Control is implemented for path tracking. We demonstrate the effectiveness of this framework in simulation, from both the decision-making and control layers. The proposed architecture also has the potential to be extended to o…
C$^2$T: Captioning-Structure and LLM-Aligned Common-Sense Reward Learning for Traffic--Vehicle Coordination
The State Key Laboratory of Internet of Things for Smart City, University of Macau（物联网智能城市国家重点实验室，澳门大学）；Wuhan University（武汉大学）；Hong Kong Polytechnic University（香港理工大学）；Computer and Information Science, University of Macau（澳门大学计算机与信息科学）
ETH Zurich（苏黎世联邦理工学院）；AIT – Austrian Institute of Technology GmbH（奥地利技术研究所）；AVL List GmbH（AVL李斯特公司）
In a future connected vehicle environment, an optimized route and motion planning should not only fulfill efficiency and safety constraints but also minimize vehicle motions and oscillations, causing poor ride comfort perceived by passengers. This work provides a framework for a large-scale and cost-efficient evaluation to address AV's ride comfort and allow the comparison of different comfort assessment strategies. The proposed tool also gives insights to comfort data, allowing for the development of novel algorithms, guidelines, or motion planning systems incorporating passenger comfort. A v…
FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院（KIT））
While motion planning approaches for automated driving often focus on safety and mathematical optimality with respect to technical parameters, they barely consider convenience, perceived safety for the passenger and comprehensibility for other traffic participants. For automated driving in mixed traffic, however, this is key to reach public acceptance. In this paper, we revise the problem statement of motion planning in mixed traffic: Instead of largely simplifying the motion planning problem to a convex optimization problem, we keep a more complex probabilistic multi agent model and strive fo…
Five AI Ltd.（Five AI有限公司）；University of Oxford（牛津大学）；University of Edinburgh（爱丁堡大学）
Achieving a proper balance between planning quality, safety and efficiency is a major challenge for autonomous driving. Optimisation-based motion planners are capable of producing safe, smooth and comfortable plans, but often at the cost of runtime efficiency. On the other hand, naively deploying trajectories produced by efficient-to-run deep imitation learning approaches might risk compromising safety. In this paper, we present PILOT -- a planning framework that comprises an imitation neural network followed by an efficient optimiser that actively rectifies the network's plan, guaranteeing fu…
