An Autonomous Driving Model Integrated with BEV-V2X Perception, Fusion Prediction of Motion and Occupancy, and Driving Planning, in Complex Traffic Intersections
Fukang Li, Wenlin Ou, Kunpeng Gao, Yuwen Pang, Yifei Li, Henry Fan
备注：
The content of the paper has not received unanimous consent from all the members and requires further evaluation prior to submission
车联网（V2X）识别的全面性丰富并整体塑造了全局鸟瞰图（BEV）感知，融合了丰富的语义并整合了驾驶场景信息，从而为车辆状态预测、决策和驾驶规划提供特征支持。利用V2X消息集构建BEV地图被证明是网联自动驾驶车辆（CAVs）的一种有效感知方法。具体而言，Map Msg.（MAP）、Signal Phase And Timing（SPAT）和Roadside Information（RSI）有助于实现道路连通性、同步交通信号导航和障碍物预警。此外，利用来自多车辆的时序Basic Safety Msg.（BSM）数据能够实现实时感知和未来状态预测。因此，本文开发了一个综合的自动驾驶模型，该模型依赖于BEV-V2X感知、基于Interacting Multiple model Unscented Kalman Filter（IMM-UKF）的融合预测，以及基于深度强化学习（DRL）的决策和规划。我们将它们集成到DRL环境中，以开发一套最优的统一驾驶行为集，涵盖避障、变道、超车、转向操作和同步交通信号导航。因此，模拟了一个复杂的交通路口场景，并将训练好的模型应用于驾驶规划。观察到的驾驶行为与经验丰富的驾驶员非常相似，表现出预期动作，并揭示了驾驶策略的显著操作亮点。
英文摘要：
The comprehensiveness of vehicle-to-everything (V2X) recognition enriches and holistically shapes the global Birds-Eye-View (BEV) perception, incorporating rich semantics and integrating driving scene information, thereby serving features of vehicle state prediction, decision-making and driving planning. Utilizing V2X message sets to form BEV map proves to be an effective perception method for connected and automated vehicles (CAVs). Specifically, Map Msg. (MAP), Signal Phase And Timing (SPAT) and Roadside Information (RSI) contributes to the achievement of road connectivity, synchronized traffic signal navigation and obstacle warning. Moreover, harnessing time-sequential Basic Safety Msg. (BSM) data from multiple vehicles allows for the real-time perception and future state prediction. Therefore, this paper develops a comprehensive autonomous driving model that relies on BEV-V2X perception, Interacting Multiple model Unscented Kalman Filter (IMM-UKF)-based fusion prediction, and deep reinforcement learning (DRL)-based decision making and planning. We integrated them into a DRL environment to develop an optimal set of unified driving behaviors that encompass obstacle avoidance, lane changes, overtaking, turning maneuver, and synchronized traffic signal navigation. Consequently, a complex traffic intersection scenario was simulated, and the well-trained model was applied for driving planning. The observed driving behavior closely resembled that of an experienced driver, exhibiting anticipatory actions and revealing notable operational highlights of driving policy.
Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe
Shanghai AI Lab（上海人工智能实验室）；SenseTime（商汤科技）；The University of Hong Kong（香港大学）；Chinese University of Hong Kong（香港中文大学）；Huawei Inc.（华为公司）；Beihang University（北京航空航天大学）
The advance towards higher levels of automation within the field of automated driving is accompanied by increasing requirements for the operational safety of vehicles. Induced by the limitation of computational resources, trade-offs between the computational complexity of algorithms and their potential to ensure safe operation of automated vehicles are often encountered. Situation-aware environment perception presents one promising example, where computational resources are distributed to regions within the perception area that are relevant for the task of the automated vehicle. While prior ma…
Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving
Donald Bren School of Information and Computer Sciences, University of California, Irvine（加州大学欧文分校唐纳德·布伦信息与计算机科学学院）；School of Computing, National University of Singapore（新加坡国立大学计算学院）；Nanyang Technological University（南洋理工大学）；Shanghai AI Laboratory（上海人工智能实验室）
FastOcc: Accelerating 3D Occupancy Prediction by Fusing the 2D Bird's-Eye View and Perspective View
School of Computer Science, Fudan University（复旦大学计算机科学技术学院）；Mogo Auto Intelligence and Telematics Information Technology Co., Ltd（摩国汽车智能与远程信息处理信息技术有限公司）；Faculty of Information Technology, Beijing University of Technology（北京工业大学信息学部）；Institute of Science and Technology for Brain-Inspired Intelligence, Fudan University（复旦大学脑智能科学与技术研究院）
Progressive Bird's Eye View Perception for Safety-Critical Autonomous Driving: A Comprehensive Survey
State Key Laboratory of Robotics and System（机器人系统国家重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）；State Key Laboratory of Intelligent Green Vehicle and Mobility（智能绿色车辆与移动性国家重点实验室）；Tsinghua University（清华大学）；the School of Mechanical and Aerospace Engineering（机械与航空航天工程学院）；Nanyang Technological University（南洋理工大学）；Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence（北京交通数据挖掘与具身智能重点实验室）；Beijing Jiaotong University（北京交通大学）；the Institute for Infocomm Research（信息通信研究所）；A*STAR；the Engineering Cluster（工程集群）；the Singapore Institute of Technology（新加坡理工学院）
Autonomously driving vehicles require a complete and robust perception of the local environment. A main challenge is to perceive any other road users, where multi-object tracking or occupancy grid maps are commonly used. The presented approach combines both methods to compensate false positives and receive a complementary environment perception. Therefore, an environment perception framework is introduced that defines a common representation, extracts objects from a dynamic occupancy grid map and fuses them with tracks of a Labeled Multi-Bernoulli filter. Finally, a confidence value is develop…
Risk Occupancy: A New and Efficient Paradigm through Vehicle-Road-Cloud Collaboration
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与出行学院）；State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University（清华大学智能绿色车辆与出行国家重点实验室）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）
Renault S.A.S（雷诺股份有限公司）；Sorbonne universités, Université de technologie de Compiègne, CNRS, Heudiasyc（索邦大学，贡比涅技术大学，法国国家科学研究中心，Heudiasyc实验室）；Monash University（莫纳什大学）
Camera-based end-to-end driving neural networks bring the promise of a low-cost system that maps camera images to driving control commands. These networks are appealing because they replace laborious hand engineered building blocks but their black-box nature makes them difficult to delve in case of failure. Recent works have shown the importance of using an explicit intermediate representation that has the benefits of increasing both the interpretability and the accuracy of networks' decisions. Nonetheless, these camera-based networks reason in camera view where scale is not homogeneous and he…
This paper proposes a perception and path planning pipeline for autonomous racing in an unknown bounded course. The pipeline was initially created for the 2021 evGrandPrix autonomous division and was further improved for the 2022 event, both of which resulting in first place finishes. Using a simple LiDAR-based perception pipeline feeding into an occupancy grid based expansion algorithm, we determine a goal point to drive. This pipeline successfully achieved reliable and consistent laps in addition with occupancy grid algorithm to know the ways around a cone-defined track with an averaging spe…
RoboBEV: Towards Robust Bird's Eye View Perception under Corruptions
Huazhong University of Science and Technology（华中科技大学）；Shanghai AI Laboratory（上海人工智能实验室）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）
Trans2Occ: Voxel Occupancy Estimation and Grasp for Transparent Objects from Simulation to Reality
Shanghai AI Laboratory（上海人工智能实验室）；SUSTech（南方科技大学）；CUHK（香港中文大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Oxford（牛津大学）；Beihang University（北京航空航天大学）；Nanyang Technological University（南洋理工大学）；University of Science and Technology of China（中国科学技术大学）
IBM T. J. Watson Research Center（IBM T. J. 沃森研究中心）
In this paper we provide an overview of a new framework for robot perception, real-world modelling, and navigation that uses a stochastic tesselated representation of spatial information called the Occupancy Grid. The Occupancy Grid is a multi-dimensional random field model that maintains probabilistic estimates of the occupancy state of each cell in a spatial lattice. Bayesian estimation mechanisms employing stochastic sensor models allow incremental updating of the Occupancy Grid using multi-view, multi-sensor data, composition of multiple maps, decision-making, and incorporation of robot an…
