About 30% of all traffic crash fatalities in the United States involve drunk drivers, making the prevention of drunk driving paramount to vehicle safety in the US and other locations which have a high prevalence of driving while under the influence of alcohol. Driving impairment can be monitored through active use of sensors (when drivers are asked to engage in providing breath samples to a vehicle instrument or when pulled over by a police officer), but a more passive and robust mechanism of sensing may allow for wider adoption and benefit of intelligent systems that reduce drunk driving accidents. This could assist in identifying impaired drivers before they drive, or early in the driving process (before a crash or detection by law enforcement). In this research, we introduce a study which adopts a multi-modal ensemble of visual, thermal, audio, and chemical sensors to (1) examine the impact of acute alcohol administration on driving performance in a driving simulator, and (2) identify data-driven methods for detecting driving under the influence of alcohol. We describe computer vision and machine learning models for analyzing the driver's face in thermal imagery, and introduce a pipeline for training models on data collected from drivers with a range of breath-alcohol content levels, including discussion of relevant machine learning phenomena which can help in future experiment design for related studies.
Driver attention prediction is currently becoming the focus in safe driving research community, such as the DR(eye)VE project and newly emerged Berkeley DeepDrive Attention (BDD-A) database in critical situations. In safe driving, an essential task is to predict the incoming accidents as early as possible. BDD-A was aware of this problem and collected the driver attention in laboratory because of the rarity of such scenes. Nevertheless, BDD-A focuses the critical situations which do not encounter actual accidents, and just faces the driver attention prediction task, without a close step for ac…
Driving as a Diagnostic Tool: Scenario-based Cognitive Assessment in Older Drivers from Driving Video
organization= Department of Electrical \& Computer Engineering , addressline= Iowa State University , city= Ames , state= IA , postcode= 50011 , country= USA；organization= Institute for Transportation , addressline= Iowa State University , city= Ames , state= IA , postcode= 50010 , country= USA；Brain Health Lab , addressline= University of Nebraska Medical Center , city= Omaha , state= NE , postcode= 68198 , country= USA；organization= Department of Mechanical Engineering , addressline= Iowa State University , city= Ames , state= IA , postcode= 50011 , country= USA
Naturalistic driving data (NDD) is an important source of information to understand crash causation and human factors and to further develop crash avoidance countermeasures. Videos recorded while driving are often included in such datasets. While there is often a large amount of video data in NDD, only a small portion of them can be annotated by human coders and used for research, which underuses all video data. In this paper, we explored a computer vision method to automatically extract the information we need from videos. More specifically, we developed a 3D ConvNet algorithm to automaticall…
In this paper, we study learning generalized driving style representations from automobile GPS trip data. We propose a novel Autoencoder Regularized deep neural Network (ARNet) and a trip encoding framework trip2vec to learn drivers' driving styles directly from GPS records, by combining supervised and unsupervised feature learning in a unified architecture. Experiments on a challenging driver number estimation problem and the driver identification problem show that ARNet can learn a good generalized driving style representation: It significantly outperforms existing methods and alternative ar…
Chongqing University（重庆大学）；University of Waterloo（滑铁卢大学）；Shenzhen University（深圳大学）
In this paper, we introduce a new dataset, the driver emotion facial expression (DEFE) dataset, for driver spontaneous emotions analysis. The dataset includes facial expression recordings from 60 participants during driving. After watching a selected video-audio clip to elicit a specific emotion, each participant completed the driving tasks in the same driving scenario and rated their emotional responses during the driving processes from the aspects of dimensional emotion and discrete emotion. We also conducted classification experiments to recognize the scales of arousal, valence, dominance, …
Boston University（波士顿大学）；Honda Research Institute USA（美国本田研究院）
Driving Scene understanding is a key ingredient for intelligent transportation systems. To achieve systems that can operate in a complex physical and social environment, they need to understand and learn how humans drive and interact with traffic scenes. We present the Honda Research Institute Driving Dataset (HDD), a challenging dataset to enable research on learning driver behavior in real-life environments. The dataset includes 104 hours of real human driving in the San Francisco Bay Area collected using an instrumented vehicle equipped with different sensors. We provide a detailed analysis…
Numerous car accidents are caused by improper driving maneuvers. Serious injuries are however avoidable if such driving maneuvers are detected beforehand and the driver is assisted accordingly. In fact, various recent research has focused on the automated prediction of driving maneuver based on hand-crafted features extracted mainly from in-cabin driver videos. Since the outside view from the traffic scene may also contain informative features for driving maneuver prediction, we present a framework for the detection of the drivers' intention based on both in-cabin and traffic scene videos. Mor…
Ohio State University（俄亥俄州立大学）；Lyft Inc.（Lyft公司）；Google Inc.（谷歌公司）；Nationwide Mutual Insurance Co.（全美互惠保险公司）
Identifying driving styles is the task of analyzing the behavior of drivers in order to capture variations that will serve to discriminate different drivers from each other. This task has become a prerequisite for a variety of applications, including usage-based insurance, driver coaching, driver action prediction, and even in designing autonomous vehicles; because driving style encodes essential information needed by these applications. In this paper, we present a deep-neural-network architecture, we term D-CRNN, for building high-fidelity representations for driving style, that combine the p…
In this paper, we presented a preliminary study for tactical driver behavior detection from untrimmed naturalistic driving recordings. While supervised learning based detection is a common approach, it suffers when labeled data is scarce. Manual annotation is both time-consuming and expensive. To emphasize this problem, we experimented on a 104-hour real-world naturalistic driving dataset with a set of predefined driving behaviors annotated. There are three challenges in the dataset. First, predefined driving behaviors are sparse in a naturalistic driving setting. Second, the distribution of d…
Robust driver attention prediction for critical situations is a challenging computer vision problem, yet essential for autonomous driving. Because critical driving moments are so rare, collecting enough data for these situations is difficult with the conventional in-car data collection protocol---tracking eye movements during driving. Here, we first propose a new in-lab driver attention collection protocol and introduce a new driver attention dataset, Berkeley DeepDrive Attention (BDD-A) dataset, which is built upon braking event videos selected from a large-scale, crowd-sourced driving video …
M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与出行学院）；SenseTime Research（商汤科技研究院）；School of Mechanical Engineering, Beijing Institute of Technology（北京理工大学机械工程学院）；School of Computer and Information Technology, Beijing Jiaotong University（北京交通大学计算机与信息技术学院）
Beijing Institute of Technology（北京理工大学）；University of California, Berkeley（加州大学伯克利分校）；University of Michigan（密歇根大学）
Analysis and recognition of driving styles are profoundly important to intelligent transportation and vehicle calibration. This paper presents a novel driving style analysis framework using the primitive driving patterns learned from naturalistic driving data. In order to achieve this, first, a Bayesian nonparametric learning method based on a hidden semi-Markov model (HSMM) is introduced to extract primitive driving patterns from time series driving data without prior knowledge of the number of these patterns. In the Bayesian nonparametric approach, we utilize a hierarchical Dirichlet process…
A Contextual Analysis of Driver-Facing and Dual-View Video Inputs for Distraction Detection in Naturalistic Driving Environments
Environmental Engineering The University of Memphis Memphis, TN, USA Email（环境工程 明尼苏达大学 内华达州法戈 邮箱）；Environmental Engineering North Dakota State University Fargo, ND, USA Email（环境工程 内华达州立大学 内华达州法戈 邮箱）
Driver attention prediction is becoming an essential research problem in human-like driving systems. This work makes an attempt to predict the driver attention in driving accident scenarios (DADA). However, challenges tread on the heels of that because of the dynamic traffic scene, intricate and imbalanced accident categories. In this work, we design a semantic context induced attentive fusion network (SCAFNet). We first segment the RGB video frames into the images with different semantic regions (i.e., semantic images), where each region denotes one kind of semantic categories of the scene (e…
Driver Age and Its Effect on Key Driving Metrics: Insights from Dynamic Vehicle Data
Iowa State University of Science and Technology（爱荷华州立科技大学）；National Institute on Aging（国家老龄化研究所）；Toyota Motor Engineering & Manufacturing North America Inc.（丰田北美汽车工程制造公司）
Distracted drivers are more likely to fail to anticipate hazards, which result in car accidents. Therefore, detecting anomalies in drivers' actions (i.e., any action deviating from normal driving) contains the utmost importance to reduce driver-related accidents. However, there are unbounded many anomalous actions that a driver can do while driving, which leads to an 'open set recognition' problem. Accordingly, instead of recognizing a set of anomalous actions that are commonly defined by previous dataset providers, in this work, we propose a contrastive learning approach to learn a metric to …
NARRATE: A Multimodal Real-World Australian Driving Dataset for Human-Centred Explanations in Automated Driving
ARC Training Centre for Automated Vehicles in Rural and Remote Regions (AVR3)（澳大利亚农村及偏远地区自动驾驶车辆ARC培训中心）；Université Gustave Eiffel（Gustave Eiffel大学）；Queensland University of Technology (QUT)（昆士兰科技大学）
Driving is a routine activity for many, but it is far from simple. Drivers deal with multiple concurrent tasks, such as keeping the vehicle in the lane, observing and anticipating the actions of other road users, reacting to hazards, and dealing with distractions inside and outside the vehicle. Failure to notice and respond to the surrounding objects and events can cause accidents. The ongoing improvements of the road infrastructure and vehicle mechanical design have made driving safer overall. Nevertheless, the problem of driver inattention has remained one of the primary causes of accidents.…
Purdue University（普渡大学）；Honda Research Institute USA（本田研究所美国）
A significant amount of people die in road accidents due to driver errors. To reduce fatalities, developing intelligent driving systems assisting drivers to identify potential risks is in an urgent need. Risky situations are generally defined based on collision prediction in the existing works. However, collision is only a source of potential risks, and a more generic definition is required. In this work, we propose a novel driver-centric definition of risk, i.e., objects influencing drivers' behavior are risky. A new task called risk object identification is introduced. We formulate the task …
In order to increase road safety, among the visual and manual distractions, modern intelligent vehicles need also to detect cognitive distracted driving (i.e., the drivers mind wandering). In this study, the influence of cognitive processes on the drivers gaze behavior is explored. A novel image-based representation of the driver's eye-gaze dispersion is proposed to estimate cognitive distraction. Data are collected on open highway roads, with a tailored protocol to create cognitive distraction. The visual difference of created shapes shows that a driver explores a wider area in neutral drivin…
This paper introduces the autonomous system of the "Smart Shark II" which won the Formula Student Autonomous China (FSAC) Competition in 2018. In this competition, an autonomous racecar is required to complete autonomously two laps of unknown track. In this paper, the author presents the self-driving software structure of this racecar which ensure high vehicle speed and safety. The key components ensure a stable driving of the racecar, LiDAR-based and Vision-based cone detection provide a redundant perception; the EKF-based localization offers high accuracy and high frequency state estimation;…
Waymo；Zoox；Georgia Institute of Technology（佐治亚理工学院）
This work proposes a new self-driving framework that uses a human driver control model, whose feature-input values are extracted from images using deep convolutional neural networks (CNNs). The development of image processing techniques using CNNs along with accelerated computing hardware has recently enabled real-time detection of these feature-input values. The use of human driver models can lead to more "natural" driving behavior of self-driving vehicles. Specifically, we use the well-known two-point visual driver control model as the controller, and we use a top-down lane cost map CNN and …
Mind2Drive: Predicting Driver Intentions from EEG in Real-world On-Road Driving
Department of Computer Science, Imam Mohammad Ibn Saud Islamic University（伊玛姆·莫哈默德·伊本·沙特伊斯兰大学计算机科学系）；Department of Computer Science, Princess Nourah bint Abdulrahman University（努拉·本·阿卜杜勒拉赫曼大学计算机科学系）
Massachusetts Institute of Technology (MIT)（麻省理工学院（MIT））；Touchstone Evaluations, Inc.（托奇斯通评估公司）
For the foreseeble future, human beings will likely remain an integral part of the driving task, monitoring the AI system as it performs anywhere from just over 0% to just under 100% of the driving. The governing objectives of the MIT Autonomous Vehicle Technology (MIT-AVT) study are to (1) undertake large-scale real-world driving data collection that includes high-definition video to fuel the development of deep learning based internal and external perception systems, (2) gain a holistic understanding of how human beings interact with vehicle automation technology by integrating video data wi…
