Ensuring traffic safety and mitigating accidents in modern driving is of paramount importance, and computer vision technologies have the potential to significantly contribute to this goal. This paper presents a multi-modal Vision Transformer for Driver Distraction Detection (termed ViT-DD), which incorporates inductive information from training signals related to both distraction detection and driver emotion recognition. Additionally, a self-learning algorithm is developed, allowing for the seamless integration of driver data without emotion labels into the multi-task training process of ViT-DD. Experimental results reveal that the proposed ViT-DD surpasses existing state-of-the-art methods for driver distraction detection by 6.5% and 0.9% on the SFDDD and AUCDD datasets, respectively.
Driver inattention assessment has become a very active field in intelligent transportation systems. Based on active sensor Kinect and computer vision tools, we have built an efficient module for detecting driver distraction and recognizing the type of distraction. Based on color and depth map data from the Kinect, our system is composed of four sub-modules. We call them eye behavior (detecting gaze and blinking), arm position (is the right arm up, down, right of forward), head orientation, and facial expressions. Each module produces relevant information for assessing driver inattention. They …
University of Arizona（亚利桑那大学）；Cornell University（康奈尔大学）
To help prevent motor vehicle accidents, there has been significant interest in finding an automated method to recognize signs of driver distraction, such as talking to passengers, fixing hair and makeup, eating and drinking, and using a mobile phone. In this paper, we present an automated supervised learning method called Drive-Net for driver distraction detection. Drive-Net uses a combination of a convolutional neural network (CNN) and a random decision forest for classifying images of a driver. We compare the performance of our proposed Drive-Net to two other popular machine-learning approa…
This study proposes an approach to segment human object from a depth image based on histogram of depth values. The region of interest is first extracted based on a predefined threshold for histogram regions. A region growing process is then employed to separate multiple human bodies with the same depth interval. Our contribution is the identification of an adaptive growth threshold based on the detected histogram region. To demonstrate the effectiveness of the proposed method, an application in driver distraction detection was introduced. After successfully extracting the driver's position ins…
Central Electronics Engineering Research Institute (CSIR-CEERI)（中央电子工程研究所）
As the proportion of road accidents increases each year, driver distraction continues to be an important risk component in road traffic injuries and deaths. The distractions caused by the increasing use of mobile phones and other wireless devices pose a potential risk to road safety. Our current study aims to aid the already existing techniques in driver posture recognition by improving the performance in the driver distraction classification problem. We present an approach using a genetic algorithm-based ensemble of six independent deep neural architectures, namely, AlexNet, VGG-16, Efficient…
A Contextual Analysis of Driver-Facing and Dual-View Video Inputs for Distraction Detection in Naturalistic Driving Environments
Environmental Engineering The University of Memphis Memphis, TN, USA Email（环境工程 明尼苏达大学 内华达州法戈 邮箱）；Environmental Engineering North Dakota State University Fargo, ND, USA Email（环境工程 内华达州立大学 内华达州法戈 邮箱）
In order to increase road safety, among the visual and manual distractions, modern intelligent vehicles need also to detect cognitive distracted driving (i.e., the drivers mind wandering). In this study, the influence of cognitive processes on the drivers gaze behavior is explored. A novel image-based representation of the driver's eye-gaze dispersion is proposed to estimate cognitive distraction. Data are collected on open highway roads, with a tailored protocol to create cognitive distraction. The visual difference of created shapes shows that a driver explores a wider area in neutral drivin…
Great Learning/Northwestern University（Great Learning/西北大学）；Great Learning
With over 50 million car sales annually and over 1.3 million deaths every year due to motor accidents we have chosen this space. India accounts for 11 per cent of global death in road accidents. Drivers are held responsible for 78% of accidents. Road safety problems in developing countries is a major concern and human behavior is ascribed as one of the main causes and accelerators of road safety problems. Driver distraction has been identified as the main reason for accidents. Distractions can be caused due to reasons such as mobile usage, drinking, operating instruments, facial makeup, social…
Distracted driving is deadly, claiming 3,477 lives in the U.S. in 2015 alone. Although there has been a considerable amount of research on modeling the distracted behavior of drivers under various conditions, accurate automatic detection using multiple modalities and especially the contribution of using the speech modality to improve accuracy has received little attention. This paper introduces a new multimodal dataset for distracted driving behavior and discusses automatic distraction detection using features from three modalities: facial expression, speech and car signals. Detailed multimoda…
The American University in Cairo（开罗美国大学）；Technical University of Munich（慕尼黑工业大学）；Ain Shams University（艾因夏姆斯大学）
The World Health Organization (WHO) reported 1.25 million deaths yearly due to road traffic accidents worldwide and the number has been continuously increasing over the last few years. Nearly fifth of these accidents are caused by distracted drivers. Existing work of distracted driver detection is concerned with a small set of distractions (mostly, cell phone usage). Unreliable ad-hoc methods are often used.In this paper, we present the first publicly available dataset for driver distraction identification with more distraction postures than existing alternatives. In addition, we propose a rel…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The task of driver attention prediction has drawn considerable interest among researchers in robotics and the autonomous vehicle industry. Driver attention prediction can play an instrumental role in mitigating and preventing high-risk events, like collisions and casualties. However, existing driver attention prediction models neglect the distraction state and intention of the driver, which can significantly influence how they observe their surroundings. To address these issues, we present a new driver attention dataset, CoCAtt (Cognitive-Conditioned Attention). Unlike previous driver attentio…
Smart Eye（Smart Eye公司）；Massachusetts Institute of Technology（麻省理工学院）；SureStart（SureStart公司）
Distracted drivers are dangerous drivers. Equipping advanced driver assistance systems (ADAS) with the ability to detect driver distraction can help prevent accidents and improve driver safety. In order to detect driver distraction, an ADAS must be able to monitor their visual attention. We propose a model that takes as input a patch of the driver's face along with a crop of the eye-region and classifies their glance into 6 coarse regions-of-interest (ROIs) in the vehicle. We demonstrate that an hourglass network, trained with an additional reconstruction loss, allows the model to learn strong…
University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Laboratory for Intelligent and Safe Automobiles（智能与安全汽车实验室）
The intelligent vehicle community has devoted considerable efforts to model driver behavior, and in particular to detect and overcome driver distraction in an effort to reduce accidents caused by driver negligence. However, as the domain increasingly shifts towards autonomous and semi-autonomous solutions, the driver is no longer integral to the decision making process, indicating a need to refocus efforts elsewhere. To this end, we propose to study pedestrian distraction instead. In particular, we focus on detecting pedestrians who are engaged in secondary activities involving their cellphone…
Manual (hand-related) activity is a significant source of crash risk while driving. Accordingly, analysis of hand position and hand activity occupation is a useful component to understanding a driver's readiness to take control of a vehicle. Visual sensing through cameras provides a passive means of observing the hands, but its effectiveness varies depending on camera location. We introduce an algorithmic framework, SMART Hands, for accurate hand classification with an ensemble of camera views using machine learning. We illustrate the effectiveness of this framework in a 4-camera setup, reachi…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
The task of driver attention prediction has drawn considerable interest among researchers in robotics and the autonomous vehicle industry. Driver attention prediction can play an instrumental role in mitigating and preventing high-risk events, like collisions and casualties. However, existing driver attention prediction models neglect the distraction state and intention of the driver, which can significantly influence how they observe their surroundings. To address these issues, we present a new driver attention dataset, CoCAtt (Cognitive-Conditioned Attention). Unlike previous driver attentio…
Countless traffic accidents often occur because of the inattention of the drivers. Many factors can contribute to distractions while driving, since objects or events to physiological conditions, as drowsiness and fatigue, do not allow the driver to stay attentive. The technological progress allowed the development and application of many solutions to detect the attention in real situations, promoting the interest of the scientific community in these last years. Commonly, these solutions identify the lack of attention and alert the driver, in order to help her/him to recover the attention, avoi…
Vicomtech Foundation, Basque Research and Technology Alliance (BRTA)（维科姆技术基金会，巴斯克研究与技术联盟）；ETS Ingenieros de Telecomunicación, Universidad Politécnica de Madrid（马德里理工大学电信工程学院）；Dependability Research Lab, Intel Labs Europe, Intel Deutschland GmbH（英特尔德国有限公司欧洲英特尔实验室可靠性研究实验室）
Vision is the richest and most cost-effective technology for Driver Monitoring Systems (DMS), especially after the recent success of Deep Learning (DL) methods. The lack of sufficiently large and comprehensive datasets is currently a bottleneck for the progress of DMS development, crucial for the transition of automated driving from SAE Level-2 to SAE Level-3. In this paper, we introduce the Driver Monitoring Dataset (DMD), an extensive dataset which includes real and simulated driving scenarios: distraction, gaze allocation, drowsiness, hands-wheel interaction and context data, in 41 hours of…
Understanding driver activity is vital for in-vehicle systems that aim to reduce the incidence of car accidents rooted in cognitive distraction. Automating real-time behavior recognition while ensuring actions classification with high accuracy is however challenging, given the multitude of circumstances surrounding drivers, the unique traits of individuals, and the computational constraints imposed by in-vehicle embedded platforms. Prior work fails to jointly meet these runtime/accuracy requirements and mostly rely on a single sensing modality, which in turn can be a single point of failure. I…
AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perception
Fudan University（复旦大学）；Institute of Meta-Medical（元医学研究所）；Engineering Research Center of AI and Robotics, Ministry of Education（教育部人工智能与机器人工程研究中心）；AI and Unmanned Systems Engineering Research Center of Jilin Province（吉林省人工智能与无人系统工程研究中心）；Boli Technology Co., Ltd.（博力科技有限公司）
Traditional video-based human activity recognition has experienced remarkable progress linked to the rise of deep learning, but this effect was slower as it comes to the downstream task of driver behavior understanding. Understanding the situation inside the vehicle cabin is essential for Advanced Driving Assistant System (ADAS) as it enables identifying distraction, predicting driver's intent and leads to more convenient human-vehicle interaction. At the same time, driver observation systems face substantial obstacles as they need to capture different granularities of driver states, while the…
ALERT Open Dataset and Input-Size-Agnostic Vision Transformer for Driver Activity Recognition using IR-UWB
Department of ICE, Hanbat National University, Daejeon, Republic of Korea（信息电子工程系，翰 bam国立大学）；Department of ECE and INMC, Seoul National University, Seoul, Republic of Korea（电子工程系和智能纳米材料中心，首尔国立大学）；Department of ECE and INMC, Seoul National University（电子工程系和智能纳米材料中心，首尔国立大学）
University of Tehran（德黑兰大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）；Henry Ford Health System（亨利福特医疗系统）
Heart rate and blink duration are two vital physiological signals which give information about cardiac activity and consciousness. Monitoring these two signals is crucial for various applications such as driver drowsiness detection. As there are several problems posed by the conventional systems to be used for continuous, long-term monitoring, a remote blink and ECG monitoring system can be used as an alternative. For estimating the blink duration, two strategies are used. In the first approach, pictures of open and closed eyes are fed into an Artificial Neural Network (ANN) to decide whether …
Enhancing Cross-Dataset Performance of Distracted Driving Detection With Score Softmax Classifier And Dynamic Gaussian Smoothing Supervision
State Key Laboratory of Advanced Design and Manufacturing for Vehicle Body（汽车车身先进设计与制造国家重点实验室）；College of Mechanical and Vehicle Engineering, Hunan University（湖南大学机械与运载工程学院）；Faculty of Engineering and IT, University of Technology Sydney（悉尼科技大学工程与信息技术学院）；College of Automotive and Mechanical Engineering, ChangSha University of Science Technology（长沙理工大学汽车与机械工程学院）
Institute for Advanced Studies in Basic Sciences (IASBS)（基础科学高级研究学院）；Charles University（查理大学）；Institute for Research in Fundamental Sciences (IPM)（基础科学研究所）
Nowadays, automobile manufacturers make efforts to develop ways to make cars fully safe. Monitoring driver's actions by computer vision techniques to detect driving mistakes in real-time and then planning for autonomous driving to avoid vehicle collisions is one of the most important issues that has been investigated in the machine vision and Intelligent Transportation Systems (ITS). The main goal of this study is to prevent accidents caused by fatigue, drowsiness, and driver distraction. To avoid these incidents, this paper proposes an integrated safety system that continuously monitors the d…
RGB-D salient object detection (SOD) is usually formulated as a problem of classification or regression over two modalities, i.e., RGB and depth. Hence, effective RGBD feature modeling and multi-modal feature fusion both play a vital role in RGB-D SOD. In this paper, we propose a depth-sensitive RGB feature modeling scheme using the depth-wise geometric prior of salient objects. In principle, the feature modeling scheme is carried out in a depth-sensitive attention module, which leads to the RGB feature enhancement as well as the background distraction reduction by capturing the depth geometry…
A 20% rise in car crashes in 2021 compared to 2020 has been observed as a result of increased distraction and drowsiness. Drowsy and distracted driving are the cause of 45% of all car crashes. As a means to decrease drowsy and distracted driving, detection methods using computer vision can be designed to be low-cost, accurate, and minimally invasive. This work investigated the use of the vision transformer to outperform state-of-the-art accuracy from 3D-CNNs. Two separate transformers were trained for drowsiness and distractedness. The drowsy video transformer model was trained on the National…
Search-based Testing of Vision Language Models for In-Car Scene Understanding
BMW Group, Technical University of Munich（宝马集团、慕尼黑技术大学）；Technical University of Munich（慕尼黑技术大学）；Technical University of Munich, fortiss GmbH（慕尼黑技术大学、fortiss GmbH）
University of Texas at Dallas（德克萨斯大学达拉斯分校）；Columbia University（哥伦比亚大学）
With the rapid development of intelligent vehicles and Advanced Driver-Assistance Systems (ADAS), a new trend is that mixed levels of human driver engagements will be involved in the transportation system. Therefore, necessary visual guidance for drivers is vitally important under this situation to prevent potential risks. To advance the development of visual guidance systems, we introduce a novel vision-cloud data fusion methodology, integrating camera image and Digital Twin information from the cloud to help intelligent vehicles make better decisions. Target vehicle bounding box is drawn and…
Self-Supervised Vision Transformers for CBCT-Based Detection of Temporomandibular Joint Osteoarthritis
Herman Ostrow School of Dentistry, University of Southern California（南加州大学赫尔曼·奥斯特罗牙科学院）；Viterbi School of Engineering, University of Southern California（南加州大学维特比工程学院）
