Traffic accident anticipation aims to predict accidents from dashcam videos as early as possible, which is critical to safety-guaranteed self-driving systems. With cluttered traffic scenes and limited visual cues, it is of great challenge to predict how long there will be an accident from early observed frames. Most existing approaches are developed to learn features of accident-relevant agents for accident anticipation, while ignoring the features of their spatial and temporal relations. Besides, current deterministic deep neural networks could be overconfident in false predictions, leading to high risk of traffic accidents caused by self-driving systems. In this paper, we propose an uncertainty-based accident anticipation model with spatio-temporal relational learning. It sequentially predicts the probability of traffic accident occurrence with dashcam videos. Specifically, we propose to take advantage of graph convolution and recurrent networks for relational feature learning, and leverage Bayesian neural networks to address the intrinsic variability of latent relational representations. The derived uncertainty-based ranking loss is found to significantly boost model performance by improving the quality of relational features. In addition, we collect a new Car Crash Dataset (CCD) for traffic accident anticipation which contains environmental attributes and accident reasons annotations. Experimental results on both public and the newly-compiled datasets show state-of-the-art performance of our model. Our code and CCD dataset are available at https://github.com/Cogito2012/UString.
Traffic accident anticipation aims to accurately and promptly predict the occurrence of a future accident from dashcam videos, which is vital for a safety-guaranteed self-driving system. To encourage an early and accurate decision, existing approaches typically focus on capturing the cues of spatial and temporal context before a future accident occurs. However, their decision-making lacks visual explanation and ignores the dynamic interaction with the environment. In this paper, we propose Deep ReInforced accident anticipation with Visual Explanation, named DRIVE. The method simulates both the…
Domain-Enhanced Dual-Branch Model for Efficient and Interpretable Accident Anticipation
State Key Laboratory of Internet of Things for Smart City（物联网智能城市国家重点实验室）；University of Macau（澳门大学）；Department of Civil Engineering（土木工程系）；Department of Computer and Information Science（计算机与信息科学系）；College of Transportation Engineering（交通工程学院）
Real-time Accident Anticipation for Autonomous Driving Through Monocular Depth-Enhanced 3D Modeling
University of Macau（澳门大学）；University of Electronic Science and Technology of China（电子科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；New York University（纽约大学）；Central South University（中南大学）；Beihang University（北京航空航天大学）；University of Hawaii（夏威夷大学）
Traffic accident anticipation is a vital function of Automated Driving Systems (ADSs) for providing a safety-guaranteed driving experience. An accident anticipation model aims to predict accidents promptly and accurately before they occur. Existing Artificial Intelligence (AI) models of accident anticipation lack a human-interpretable explanation of their decision-making. Although these models perform well, they remain a black-box to the ADS users, thus difficult to get their trust. To this end, this paper presents a Gated Recurrent Unit (GRU) network that learns spatio-temporal relational fea…
For survival, a living agent must have the ability to assess risk (1) by temporally anticipating accidents before they occur, and (2) by spatially localizing risky regions in the environment to move away from threats. In this paper, we take an agent-centric approach to study the accident anticipation and risky region localization tasks. We propose a novel soft-attention Recurrent Neural Network (RNN) which explicitly models both spatial and appearance-wise non-linear interaction between the agent triggering the event and another agent or static-region involved. In order to test our proposed me…
National Institute of Advanced Industrial Science and Technology (AIST)（国立先进工业科学技术研究所（AIST））；Keio University（庆应义塾大学）
In this paper, we propose a novel approach for traffic accident anticipation through (i) Adaptive Loss for Early Anticipation (AdaLEA) and (ii) a large-scale self-annotated incident database for anticipation. The proposed AdaLEA allows a model to gradually learn an earlier anticipation as training progresses. The loss function adaptively assigns penalty weights depending on how early the model can an- ticipate a traffic accident at each epoch. Additionally, we construct a Near-miss Incident DataBase for anticipation. This database contains an enormous number of traffic near- miss incident vide…
RiskProp: Collision-Anchored Self-Supervised Risk Propagation for Early Accident Anticipation
School of Computer Science, Wuhan University（武汉大学计算机学院）；Zhongguancun Academy（中关村学院）；Didi Chuxing（滴滴出行）；University of Science and Technology of China（中国科学技术大学）
CARA: Concept-Aware Risk Attention for Interpretable Collision Anticipation
Shanghai Jiao Tong University（上海交通大学）；The University of Hong Kong（香港大学）；Fudan University（复旦大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Pennsylvania（宾夕法尼亚大学）；Renmin University of China（中国人民大学）
Anticipation of accidents ahead of time in autonomous and non-autonomous vehicles aids in accident avoidance. In order to recognize abnormal events such as traffic accidents in a video sequence, it is important that the network takes into account interactions of objects in a given frame. We propose a novel Feature Aggregation (FA) block that refines each object's features by computing a weighted sum of the features of all objects in a frame. We use FA block along with Long Short Term Memory (LSTM) network to anticipate accidents in the video sequences. We report mean Average Precision (mAP) an…
In this paper, we propose a novel approach to predict group activities given the beginning frames with incomplete activity executions. Existing action prediction approaches learn to enhance the representation power of the partial observation. However, for group activity prediction, the relation evolution of people's activity and their positions over time is an important cue for predicting group activity. To this end, we propose a sequential relational anticipation model (SRAM) that summarizes the relational dynamics in the partial observation and progressively anticipates the group representat…
Deep Learning Advances in Vision-Based Traffic Accident Anticipation: A Comprehensive Review of Methods, Datasets, and Future Directions
School of Advanced Manufacturing, Fuzhou University（福州大学先进制造学院）；National Local United Engineering Lab of Flat Panel Display Technology, Fuzhou University（福州大学平板显示技术国家地方联合工程实验室）；College of Computer Software, Hanyang University（汉阳大学计算机软件学院）；College of Computing, City University of Hong Kong（香港城市大学计算学院）
The rapid advancement of sensor technologies and artificial intelligence are creating new opportunities for traffic safety enhancement. Dashboard cameras (dashcams) have been widely deployed on both human driving vehicles and automated driving vehicles. A computational intelligence model that can accurately and promptly predict accidents from the dashcam video will enhance the preparedness for accident prevention. The spatial-temporal interaction of traffic agents is complex. Visual cues for predicting a future accident are embedded deeply in dashcam video data. Therefore, the early anticipati…
World Model-Based End-to-End Scene Generation for Accident Anticipation in Autonomous Driving
State Key Laboratory of Internet of Things for Smart City（物联网智能城市国家重点实验室）；University of Macau（澳门大学）；Department of Civil Engineering（土木工程系）；Department of Computer and Information Science（计算机与信息科学系）
EQ-TAA: Equivariant Traffic Accident Anticipation via Diffusion-Based Accident Video Synthesis
Xi’an Jiaotong University（西安交通大学）；University of Macau（澳门大学）；National University of Singapore（新加坡国立大学）；Department of Electrical Engineering and Computer Science, Cleveland State University（克莱姆森大学电气工程与计算机科学系）；Institute for Infocomm Research, Agency for Science, Technology and Research (A ∗ STAR)（信息与通信研究机构，科技研究局）
Learning from the Unseen: Generative Data Augmentation for Geometric-Semantic Accident Anticipation
State Key Laboratory of Internet of Things for Smart City（物联网智能城市国家重点实验室）；University of Macau（澳门大学）；Department of Civil Engineering（土木工程系）；Department of Computer and Information Science（计算机与信息科学系）；Department of Automotive Engineering（汽车工程系）；Tsinghua University（清华大学）
Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）
Progress towards advanced systems for assisted and autonomous driving is leveraging recent advances in recognition and segmentation methods. Yet, we are still facing challenges in bringing reliable driving to inner cities, as those are composed of highly dynamic scenes observed from a moving platform at considerable speeds. Anticipation becomes a key element in order to react timely and prevent accidents. In this paper we argue that it is necessary to predict at least 1 second and we thus propose a new model that jointly predicts ego motion and people trajectories over such large time horizons…
Numerous car accidents are caused by improper driving maneuvers. Serious injuries are however avoidable if such driving maneuvers are detected beforehand and the driver is assisted accordingly. In fact, various recent research has focused on the automated prediction of driving maneuver based on hand-crafted features extracted mainly from in-cabin driver videos. Since the outside view from the traffic scene may also contain informative features for driving maneuver prediction, we present a framework for the detection of the drivers' intention based on both in-cabin and traffic scene videos. Mor…
Language-Driven Object-Oriented Two-Stage Method for Scene Graph Anticipation
Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学）；Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology（计算能力网络与信息安全重点实验室，教育部，山东计算机科学中心，齐鲁大学）；Academy of Interdisciplinary Studies, The Hong Kong University of Science and Technology（跨学科研究学院，香港科学与技术大学）
