ROAD: The ROad event Awareness Dataset for Autonomous Driving
Oxford Brookes University（牛津布鲁克斯大学）；University of Naples Federico II（那不勒斯费德里科二世大学）；ETH Zurich（苏黎世联邦理工学院）；University of Science and Technology of Mazandaran（马赞达兰科技大学）；Siemens SpA（西门子股份公司（意大利））
Gurkirt Singh, Stephen Akrigg, Manuele Di Maio, Valentina Fontana, Reza Javanmard Alitappeh, Suman Saha, Kossar Jeddisaravi, Farzad Yousefi, Jacob Culley, Tom N… 展开作者
Gurkirt Singh, Stephen Akrigg, Manuele Di Maio, Valentina Fontana, Reza Javanmard Alitappeh, Suman Saha, Kossar Jeddisaravi, Farzad Yousefi, Jacob Culley, Tom Nicholson, Jordan Omokeowa, Salman Khan, Stanislao Grazioso, Andrew Bradley, Giuseppe Di Gironimo, Fabio Cuzzolin
Humans drive in a holistic fashion which entails, in particular, understanding dynamic road events and their evolution. Injecting these capabilities in autonomous vehicles can thus take situational awareness and decision making closer to human-level performance. To this purpose, we introduce the ROad event Awareness Dataset (ROAD) for Autonomous Driving, to our knowledge the first of its kind. ROAD is designed to test an autonomous vehicle's ability to detect road events, defined as triplets composed by an active agent, the action(s) it performs and the corresponding scene locations. ROAD comprises videos originally from the Oxford RobotCar Dataset annotated with bounding boxes showing the location in the image plane of each road event. We benchmark various detection tasks, proposing as a baseline a new incremental algorithm for online road event awareness termed 3D-RetinaNet. We also report the performance on the ROAD tasks of Slowfast and YOLOv5 detectors, as well as that of the winners of the ICCV2021 ROAD challenge, which highlight the challenges faced by situation awareness in autonomous driving. ROAD is designed to allow scholars to investigate exciting tasks such as complex (road) activity detection, future event anticipation and continual learning. The dataset is available at https://github.com/gurkirt/road-dataset; the baseline can be found at https://github.com/gurkirt/3D-RetinaNet.
With the development of autonomous driving, the improvement of autonomous driving technology for individual vehicles has reached the bottleneck. The advancement of vehicle-road cooperation autonomous driving technology can expand the vehicle's perception range, supplement the perception blind area and improve the perception accuracy, to promote the development of autonomous driving technology and achieve vehicle-road integration. This project mainly uses lidar to develop data fusion schemes to realize the sharing and combination of vehicle and road equipment data and achieve the detection and …
In this paper we present a novel approach for lane detection and segmentation using generative models. Traditionally discriminative models have been employed to classify pixels semantically on a road. We model the probability distribution of lanes and road symbols by training a generative adversarial network. Based on the learned probability distribution, context-aware lanes and road signs are generated for a given image which are further quantized for nearest class label. Proposed method has been tested on BDD100K and Baidu ApolloScape datasets and performs better than state of the art and ex…
This paper presents a novel data-driven approach to vehicle motion planning and control in off-road driving scenarios. For autonomous off-road driving, environmental conditions impact terrain traversability as a function of weather, surface composition, and slope. Geographical information system (GIS) and National Centers for Environmental Information datasets are processed to provide this information for interactive planning and control system elements. A top-level global route planner (GRP) defines optimal waypoints using dynamic programming (DP). A local path planner (LPP) computes a desire…
The majority of current approaches in autonomous driving rely on High-Definition (HD) maps which detail the road geometry and surrounding area. Yet, this reliance is one of the obstacles to mass deployment of autonomous vehicles due to poor scalability of such prior maps. In this paper, we tackle the problem of online road map extraction via leveraging the sensory system aboard the vehicle itself. To this end, we design a structured model where a graph representation of the road network is generated in a hierarchical fashion within a fully convolutional network. The method is able to handle co…
Indiana University（印第安纳大学）；Honda Research Institute（本田研究所）
A vehicle driving along the road is surrounded by many objects, but only a small subset of them influence the driver's decisions and actions. Learning to estimate the importance of each object on the driver's real-time decision-making may help better understand human driving behavior and lead to more reliable autonomous driving systems. Solving this problem requires models that understand the interactions between the ego-vehicle and the surrounding objects. However, interactions among other objects in the scene can potentially also be very helpful, e.g., a pedestrian beginning to cross the roa…
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）
Road curb detection is important for autonomous driving. It can be used to determine road boundaries to constrain vehicles on roads, so that potential accidents could be avoided. Most of the current methods detect road curbs online using vehicle-mounted sensors, such as cameras or 3-D Lidars. However, these methods usually suffer from severe occlusion issues. Especially in highly-dynamic traffic environments, most of the field of view is occupied by dynamic objects. To alleviate this issue, we detect road curbs offline using high-resolution aerial images in this paper. Moreover, the detected r…
Autonomous vehicles are expected to operate safely in real-life road conditions in the next years. Nevertheless, unanticipated events such as the existence of unexpected objects in the range of the road, can put safety at risk. The advancement of sensing and communication technologies and Internet of Things may facilitate the recognition of hazardous situations and information exchange in a cooperative driving scheme, providing new opportunities for the increase of collaborative situational awareness. Safe and unobtrusive visualization of the obtained information may nowadays be enabled throug…
Northwestern University（西北大学）；Toyota Research Institute（丰田研究院）；Woven Planet
We propose a computational model to estimate a person's attended awareness of their environment. We define attended awareness to be those parts of a potentially dynamic scene which a person has attended to in recent history and which they are still likely to be physically aware of. Our model takes as input scene information in the form of a video and noisy gaze estimates, and outputs visual saliency, a refined gaze estimate, and an estimate of the person's attended awareness. In order to test our model, we capture a new dataset with a high-precision gaze tracker including 24.5 hours of gaze se…
V-SenseDrive: A Privacy-Preserving Road Video and In-Vehicle Sensor Fusion Framework for Road Safety & Driver Behaviour Modelling
School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST)（电气工程与计算机科学学院（SEECS），国立科学与技术大学（NUST））
Politecnico di Milano（米兰理工大学）；Dept. of Electronics, Information and Bioengineering（电子、信息与生物工程系）；Milano（米兰（注：此处为城市名，非机构，按要求不提取，但原数据中机构名称已包含城市信息，提取机构时以Politecnico di Milano为主））
In literature, Extended Object Tracking (EOT) algorithms developed for autonomous driving predominantly provide obstacles state estimation in cartesian coordinates in the Vehicle Reference Frame. However, in many scenarios, state representation in road-aligned curvilinear coordinates is preferred when implementing autonomous driving subsystems like cruise control, lane-keeping assist, platooning, etc. This paper proposes a Gaussian Mixture Probability Hypothesis Density~(GM-PHD) filter with an Unscented Kalman Filter~(UKF) estimator that provides obstacle state estimates in curvilinear road co…
Impact of Level 2/3 Automated Driving Technology on Road Work Zone Safety
Faculty of Railway Transportation, Shanghai Institute of Technology（上海应用技术大学轨道交通学院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）
Road-boundary detection is important for autonomous driving. It can be used to constrain autonomous vehicles running on road areas to ensure driving safety. Compared with online road-boundary detection using on-vehicle cameras/Lidars, offline detection using aerial images could alleviate the severe occlusion issue. Moreover, the offline detection results can be directly employed to annotate high-definition (HD) maps. In recent years, deep-learning technologies have been used in offline detection. But there still lacks a publicly available dataset for this task, which hinders the research progr…
Perceiving text is crucial to understand semantics of outdoor scenes and hence is a critical requirement to build intelligent systems for driver assistance and self-driving. Most of the existing datasets for text detection and recognition comprise still images and are mostly compiled keeping text in mind. This paper introduces a new "RoadText-1K" dataset for text in driving videos. The dataset is 20 times larger than the existing largest dataset for text in videos. Our dataset comprises 1000 video clips of driving without any bias towards text and with annotations for text bounding boxes and t…
RS2AD: End-to-End Autonomous Driving Data Generation from Roadside Sensor Observations
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动系统学院）；State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University（清华大学智能绿色车辆与移动系统国家重点实验室）；School of Instrumentation and Optoelectronic Engineering, BeiHang University（北航仪器与光电工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）
University of Michigan（密歇根大学）；Ford Motor Company（福特汽车公司）
We propose a novel and pragmatic framework for traffic scene perception with roadside cameras. The proposed framework covers a full-stack of roadside perception pipeline for infrastructure-assisted autonomous driving, including object detection, object localization, object tracking, and multi-camera information fusion. Unlike previous vision-based perception frameworks rely upon depth offset or 3D annotation at training, we adopt a modular decoupling design and introduce a landmark-based 3D localization method, where the detection and localization can be well decoupled so that the model can be…
Tshwane University of Technology（茨瓦内理工大学）；Central University of Technology（中央理工大学）
Autonomous driving systems are broadly used equipment in the industries and in our daily lives, they assist in production, but are majorly used for exploration in dangerous or unfamiliar locations. Thus, for a successful exploration, navigation plays a significant role. Road detection is an essential factor that assists autonomous robots achieved perfect navigation. Various techniques using camera sensors have been proposed by numerous scholars with inspiring results, but their techniques are still vulnerable to these environmental noises: rain, snow, light intensity and shadow. In addressing …
The Hong Kong University of Science and Technology（香港科技大学）；Huawei Technologies（华为技术有限公司）
Extrinsic perturbation always exists in multiple sensors. In this paper, we focus on the extrinsic uncertainty in multi-LiDAR systems for 3D object detection. We first analyze the influence of extrinsic perturbation on geometric tasks with two basic examples. To minimize the detrimental effect of extrinsic perturbation, we propagate an uncertainty prior on each point of input point clouds, and use this information to boost an approach for 3D geometric tasks. Then we extend our findings to propose a multi-LiDAR 3D object detector called MLOD. MLOD is a two-stage network where the multi-LiDAR in…
In this paper, we introduce a deep encoder-decoder network, named SalsaNet, for efficient semantic segmentation of 3D LiDAR point clouds. SalsaNet segments the road, i.e. drivable free-space, and vehicles in the scene by employing the Bird-Eye-View (BEV) image projection of the point cloud. To overcome the lack of annotated point cloud data, in particular for the road segments, we introduce an auto-labeling process which transfers automatically generated labels from the camera to LiDAR. We also explore the role of imagelike projection of LiDAR data in semantic segmentation by comparing BEV wit…
FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Tremendous progress in deep learning over the last years has led towards a future with autonomous vehicles on our roads. Nevertheless, the performance of their perception systems is strongly dependent on the quality of the utilized training data. As these usually only cover a fraction of all object classes an autonomous driving system will face, such systems struggle with handling the unexpected. In order to safely operate on public roads, the identification of objects from unknown classes remains a crucial task. In this paper, we propose a novel pipeline to detect unknown objects. Instead of …
Road extraction is an essential step in building autonomous navigation systems. Detecting road segments is challenging as they are of varying widths, bifurcated throughout the image, and are often occluded by terrain, cloud, or other weather conditions. Using just convolution neural networks (ConvNets) for this problem is not effective as it is inefficient at capturing distant dependencies between road segments in the image which is essential to extract road connectivity. To this end, we propose a Spatial and Interaction Space Graph Reasoning (SPIN) module which when plugged into a ConvNet per…
DSFC-Net: A Dual-Encoder Spatial and Frequency Co-Awareness Network for Rural Road Extraction
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；Department of Earth System Science, Tsinghua University（清华大学地球系统科学系）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；College of Information and Electrical Engineering, China Agricultural University（中国农业大学信息与电气工程学院）
CarPLAN: Context-Adaptive and Robust Planning with Dynamic Scene Awareness for Autonomous Driving
Department of Artificial Intelligence, Hanyang University（翰林大学人工智能系）；Interdisciplinary Program in Artificial Intelligence, Seoul National University（首尔国立大学人工智能跨学科项目）；Department of Electrical and Computer Engineering, Seoul National University（首尔国立大学电子与计算机工程系）；Department of Automotive Engineering, Hanyang University（翰林大学汽车工程系）
Detecting small obstacles on the road ahead is a critical part of the driving task which has to be mastered by fully autonomous cars. In this paper, we present a method based on stereo vision to reliably detect such obstacles from a moving vehicle. The proposed algorithm performs statistical hypothesis tests in disparity space directly on stereo image data, assessing freespace and obstacle hypotheses on independent local patches. This detection approach does not depend on a global road model and handles both static and moving obstacles. For evaluation, we employ a novel lost-cargo image sequen…
Oxford Robotics Institute（牛津机器人研究所）；University of Oxford（牛津大学）
Road markings provide guidance to traffic participants and enforce safe driving behaviour, understanding their semantic meaning is therefore paramount in (automated) driving. However, producing the vast quantities of road marking labels required for training state-of-the-art deep networks is costly, time-consuming, and simply infeasible for every domain and condition. In addition, training data retrieved from virtual worlds often lack the richness and complexity of the real world and consequently cannot be used directly. In this paper, we provide an alternative approach in which new road marki…
Learning Autonomy: Off-Road Navigation Enhanced by Human Input
J. Mike Walker ’66 Department of Mechanical Engineering, Texas A&M University（J. Mike Walker ’66 机械工程系，德克萨斯A&M大学）；Department of Engineering technology and Industrial Distribution, Texas A&M University（工程技术与工业分销系，德克萨斯A&M大学）
RMMDet: Road-Side Multitype and Multigroup Sensor Detection System for Autonomous Driving
Shanghai Jiao Tong University（上海交通大学）；East China Normal University（华东师范大学）；Xin Dong Interactive Entertainment Company Ltd（鑫东互动娱乐有限公司）；Summer Son Smart Technology Company Ltd（夏之子智能科技有限公司）
The task of predicting stochastic behaviors of road agents in diverse environments is a challenging problem for autonomous driving. To best understand scene contexts and produce diverse possible future states of the road agents adaptively in different environments, a prediction model should be probabilistic, multi-modal, context-driven, and general. We present Conditionalizing Variational AutoEncoders via Hypernetworks (CVAE-H); a conditional VAE that extensively leverages hypernetwork and performs generative tasks for high-dimensional problems like the prediction task. We first evaluate CVAE-…
Advancing Off-Road Autonomous Driving: The Large-Scale ORAD-3D Dataset and Comprehensive Benchmarks
Research Center for Intelligent Computing Systems, SKLP, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China, 100190（中国科学院计算技术研究所，智能计算系统研究中心，SKLP，北京，中国，100190）；Tongji University, Shanghai, China, 200092（同济大学，上海，中国，200092）；Xi’an Jiaotong University, Shaanxi, China, 710049（西安交通大学，陕西，中国，710049）；Nanchang University, Jiangxi, China, 330047（南昌大学，江西，中国，330047）；Defense Innovation Institute, Beijing, China, 100073（国防科技创新院，北京，中国，100073）
Road accidents are estimated to be the ninth leading cause of death across all age groups globally. 1.25 million people die annually from road accidents and Africa has the highest rate of road fatalities [1]. Research shows that three out of five road accidents are caused by driver-related behavioral factors [2]. Self-driving technology has the potential of saving lives lost to these preventable road accidents. Africa accounts for the majority of road fatalities and as such would benefit immensely from this technology. However, financial constraints prevent viable experimentation and research …
FZI Research Center for Information Technology（FZI信息技术研究中心）
Driving on roads is restricted by various traffic rules, aiming to ensure safety for all traffic participants. However, human road users usually do not adhere to these rules strictly, resulting in varying degrees of rule conformity. Such deviations from given rules are key components of today's road traffic. In autonomous driving, robotic agents can disturb traffic flow, when rule deviations are not taken into account. In this paper, we present an approach to derive the distribution of degrees of rule conformity from human driving data. We demonstrate our method with the Waymo Open Motion data…
Establishing Reality-Virtuality Interconnections in Urban Digital Twins for Superior Intelligent Road Inspection and Simulation
College of Electronics & Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, the State Key Laboratory of Intelligent Autonomous Systems, and Frontiers Science Center for Intelligent Autonomous Systems, Tongji University, Shanghai 201804, China（电子与信息工程学院，上海智能自主系统研究院，智能自主系统国家重点实验室，智能自主系统前沿科学中心，同济大学，上海）
IIIT Hyderabad（海得拉巴国际信息技术研究所）；Hasan Kalyoncu University（哈桑·卡尔永库大学）；UC San Diego（加州大学圣地亚哥分校）
Autonomous driving and assistance systems rely on annotated data from traffic and road scenarios to model and learn the various object relations in complex real-world scenarios. Preparation and training of deploy-able deep learning architectures require the models to be suited to different traffic scenarios and adapt to different situations. Currently, existing datasets, while large-scale, lack such diversities and are geographically biased towards mainly developed cities. An unstructured and complex driving layout found in several developing countries such as India poses a challenge to these …
Smart Roads: Roadside Perception, Vehicle-Road Cooperation and Business Model
Xidian University（西安电子科技大学）；Guangzhou Institute of Technology, Xidian University（西安电子科技大学广州研究院）；Eindhoven University of Technology（埃因霍温理工大学）；Nanyang Technological University（南洋理工大学）；University of Oslo（奥斯陆大学）；Simula Metropolitan Center for Digital Engineering（西穆拉大都会数字工程中心）
Towards All-Day Perception for Off-Road Driving: A Large-Scale Multispectral Dataset and Comprehensive Benchmark
Research Center for Intelligent Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing（智能计算系统研究中心，计算技术研究所，中国科学院，北京）；Beijing Institute of Control Engineering（北京控制工程研究所）
