In this paper, an LSTM autoencoder-based architecture is utilized for drowsiness detection with ResNet-34 as feature extractor. The problem is considered as anomaly detection for a single subject; therefore, only the normal driving representations are learned and it is expected that drowsiness representations, yielding higher reconstruction losses, are to be distinguished according to the knowledge of the network. In our study, the confidence levels of normal and anomaly clips are investigated through the methodology of label assignment such that training performance of LSTM autoencoder and interpretation of anomalies encountered during testing are analyzed under varying confidence rates. Our method is experimented on NTHU-DDD and benchmarked with a state-of-the-art anomaly detection method for driver drowsiness. Results show that the proposed model achieves detection rate of 0.8740 area under curve (AUC) and is able to provide significant improvements on certain scenarios.
Mercedes-Benz AG（梅赛德斯-奔驰股份公司）；University of Tübingen（图宾根大学）
Modern advanced driver-assistance systems analyze the driving performance to gather information about the driver's state. Such systems are able, for example, to detect signs of drowsiness by evaluating the steering or lane keeping behavior and to alert the driver when the drowsiness state reaches a critical level. However, these kinds of systems have no access to direct cues about the driver's state. Hence, the aim of this work is to extend the driver drowsiness detection in vehicles using signals of a driver monitoring camera. For this purpose, 35 features related to the driver's eye blinking…
School of Technologies, Cardiff Metropolitan University（卡迪夫城市大学技术学院）；Vel Tech Rangarajan Dr. Sagunthala R&D Institute of Science and Technology（维尔科技兰加拉詹博士萨贡塔拉科学与技术研发学院）
Human drivers have distinct driving techniques, knowledge, and sentiments due to unique driving traits. Driver drowsiness has been a serious issue endangering road safety; therefore, it is essential to design an effective drowsiness detection algorithm to bypass road accidents. Miscellaneous research efforts have been approached the problem of detecting anomalous human driver behaviour to examine the frontal face of the driver and automobile dynamics via computer vision techniques. Still, the conventional methods cannot capture complicated driver behaviour features. However, with the origin of…
Many road accidents are caused by drowsiness of the driver. While there are methods to detect closed eyes, it is a non-trivial task to detect the gradual process of a driver becoming drowsy. We consider a simple real-time detection system for drowsiness merely based on the eye blinking rate derived from the eye aspect ratio. For the eye detection we use HOG and a linear SVM. If the speed of the eye blinking drops below some empirically determined threshold, the system triggers an alarm, hence preventing the driver from falling into microsleep. In this paper, we extensively evaluate the minimal…
International Islamic University Malaysia（马来西亚国际伊斯兰大学）；University of Oklahoma（俄克拉荷马大学）；University of Memphis（孟菲斯大学）
Driver drowsiness detection using videos/images is one of the most essential areas in today's time for driver safety. The development of deep learning techniques, notably Convolutional Neural Networks (CNN), applied in computer vision applications such as drowsiness detection, has shown promising results due to the tremendous increase in technology in the recent few decades. Eyes that are closed or blinking excessively, yawning, nodding, and occlusion are all key aspects of drowsiness. In this work, we have applied four different Convolutional Neural Network (CNN) techniques on the YawDD datas…
Tongji University（同济大学）；Key Laboratory of Embedded System and Service Computing, Ministry of Education（嵌入式系统与服务计算教育部重点实验室）
Drowsiness driving is a major cause of traffic accidents and thus numerous previous researches have focused on driver drowsiness detection. Many drive relevant factors have been taken into consideration for fatigue detection and can lead to high precision, but there are still several serious constraints, such as most existing models are environmentally susceptible. In this paper, fatigue detection is considered as temporal action detection problem instead of image classification. The proposed detection system can be divided into four parts: (1) Localize the key patches of the detected driver p…
Datasets are crucial when training a deep neural network. When datasets are unrepresentative, trained models are prone to bias because they are unable to generalise to real world settings. This is particularly problematic for models trained in specific cultural contexts, which may not represent a wide range of races, and thus fail to generalise. This is a particular challenge for Driver drowsiness detection, where many publicly available datasets are unrepresentative as they cover only certain ethnicity groups. Traditional augmentation methods are unable to improve a model's performance when t…
Driver drowsiness detection has been the subject of many researches in the past few decades and various methods have been developed to detect it. In this study, as an image-based approach with adequate accuracy, along with the expedite process, we applied YOLOv3 (You Look Only Once-version3) CNN (Convolutional Neural Network) for extracting facial features automatically. Then, LSTM (Long-Short Term Memory) neural network is employed to learn driver temporal behaviors including yawning and blinking time period as well as sequence classification. To train YOLOv3, we utilized our collected datase…
For real-world driver drowsiness detection from videos, the variation of head pose is so large that the existing methods on global face is not capable of extracting effective features, such as looking aside and lowering head. Temporal dependencies with variable length are also rarely considered by the previous approaches, e.g., yawning and speaking. In this paper, we propose a Long-term Multi-granularity Deep Framework to detect driver drowsiness in driving videos containing the frontal faces. The framework includes two key components: (1) Multi-granularity Convolutional Neural Network (MCNN),…
Qatar Transportation and Traffic Safety Center, Qatar University（卡塔尔大学卡塔尔交通与交通安全中心）；College of the North Atlantic-Qatar（卡塔尔北大西洋学院）；ReDCAD Laboratory, National School of Engineers of Sfax, University of Sfax（斯法克斯大学斯法克斯国立工程学院ReDCAD实验室）
A sleepy driver is arguably much more dangerous on the road than the one who is speeding as he is a victim of microsleeps. Automotive researchers and manufacturers are trying to curb this problem with several technological solutions that will avert such a crisis. This article focuses on the detection of such micro sleep and drowsiness using neural network based methodologies. Our previous work in this field involved using machine learning with multi-layer perceptron to detect the same. In this paper, accuracy was increased by utilizing facial landmarks which are detected by the camera and that…
Council for Scientific and Industrial Research（科学与工业研究理事会）；University of Kwa-Zulu Natal（夸祖鲁-纳塔尔大学）
Convolutional Neural Networks (CNNs) have been used successfully across a broad range of areas including data mining, object detection, and in business. The dominance of CNNs follows a breakthrough by Alex Krizhevsky which showed improvements by dramatically reducing the error rate obtained in a general image classification task from 26.2% to 15.4%. In road safety, CNNs have been applied widely to the detection of traffic signs, obstacle detection, and lane departure checking. In addition, CNNs have been used in data mining systems that monitor driving patterns and recommend rest breaks when a…
University of Sfax（斯法克斯大学）；National School of Engineers (ENIS)（国家工程师学院（ENIS））
Driver drowsiness problem is considered as one of the most important reasons that increases road accidents number. We propose in this paper a new approach for realtime driver drowsiness in order to prevent road accidents. The system uses a smart video camera that takes drivers faces images and supervises the eye blink (open and close) state and head posture to detect the different drowsiness states. Face and eye detection are done by Viola and Jones technique.
Driving in a state of drowsiness is a major cause of road accidents, resulting in tremendous damage to life and property. Developing robust, automatic, real-time systems that can infer drowsiness states of drivers has the potential of making life-saving impact. However, developing drowsiness detection systems that work well in real-world scenarios is challenging because of the difficulties associated with collecting high-volume realistic drowsy data and modeling the complex temporal dynamics of evolving drowsy states. In this paper, we propose a data collection protocol that involves outfittin…
Privacy-Preserving Driver Drowsiness Detection with Spatial Self-Attention and Federated Learning
University of Canberra（堪培拉大学）；School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University（Curtin大学电子工程、计算与数学科学学院）；School of Electrical and Data Engineering, University of Technology Sydney（悉尼科技大学电子与数据工程学院）
Gwangju Institute of Science and Technology (GIST)（光州科学技术院）；Mokwon University（牧园大学）
We propose a condition-adaptive representation learning framework for the driver drowsiness detection based on 3D-deep convolutional neural network. The proposed framework consists of four models: spatio-temporal representation learning, scene condition understanding, feature fusion, and drowsiness detection. The spatio-temporal representation learning extracts features that can describe motions and appearances in video simultaneously. The scene condition understanding classifies the scene conditions related to various conditions about the drivers and driving situations such as statuses of wea…
This paper focuses on the challenge of driver safety on the road and presents a novel system for driver drowsiness detection. In this system, to detect the falling sleep state of the driver as the sign of drowsiness, Convolutional Neural Networks (CNN) are used with regarding the two goals of real-time application, including high accuracy and fastness. Three networks introduced as a potential network for eye status classifcation in which one of them is a Fully Designed Neural Network (FD-NN) and others use Transfer Learning in VGG16 and VGG19 with extra designed layers (TL-VGG). Lack of an ava…
Qatar Transportation and Traffic Safety Center, Qatar University（卡塔尔大学卡塔尔交通运输与交通安全中心）；Rutgers, The State University of New Jersey（罗格斯大学（新泽西州立大学））
Road crashes and related forms of accidents are a common cause of injury and death among the human population. According to 2015 data from the World Health Organization, road traffic injuries resulted in approximately 1.25 million deaths worldwide, i.e. approximately every 25 seconds an individual will experience a fatal crash. While the cost of traffic accidents in Europe is estimated at around 160 billion Euros, driver drowsiness accounts for approximately 100,000 accidents per year in the United States alone as reported by The American National Highway Traffic Safety Administration (NHTSA).…
Karunya Institute of Technology and Sciences（卡鲁尼亚理工学院）；Manipal Institute of Technology, Manipal Academy of Higher Education（马尼帕尔高等教育学院马尼帕尔理工学院）
Around 40 percent of accidents related to driving on highways in India occur due to the driver falling asleep behind the steering wheel. Several types of research are ongoing to detect driver drowsiness but they suffer from the complexity and cost of the models. In this paper, SleepyWheels a revolutionary method that uses a lightweight neural network in conjunction with facial landmark identification is proposed to identify driver fatigue in real time. SleepyWheels is successful in a wide range of test scenarios, including the lack of facial characteristics while covering the eye or mouth, the…
Monta Vista High School（蒙特维斯塔高中）；Cupertino High School（库比蒂诺高中）
Drowsiness on the road is a widespread problem with fatal consequences; thus, a multitude of systems and techniques have been proposed. Among existing methods, Ghoddoosian et al. utilized temporal blinking patterns to detect early signs of drowsiness, but their algorithm was tested only on a powerful desktop computer, which is not practical to apply in a moving vehicle setting. In this paper, we propose an efficient platform to run Ghoddosian's algorithm, detail the performance tests we ran to determine this platform, and explain our threshold optimization logic. After considering the Jetson N…
Driver drowsiness increases crash risk, leading to substantial road trauma each year. Drowsiness detection methods have received considerable attention, but few studies have investigated the implementation of a detection approach on a mobile phone. Phone applications reduce the need for specialised hardware and hence, enable a cost-effective roll-out of the technology across the driving population. While it has been shown that three-dimensional (3D) operations are more suitable for spatiotemporal feature learning, current methods for drowsiness detection commonly use frame-based, multi-step ap…
Blinking Beyond EAR: A Stable Eyelid Angle Metric for Driver Drowsiness Detection and Data Augmentation
Institue of Smart Engineering and Machine Elements at the Hamburg University of Technology（汉堡技术大学智能工程与机械元件研究所）；Intelligent Transport Systems group at the Australian Center for Robotics, The University of Sydney（澳大利亚机器人中心智能交通系统组）
Drowsiness can put lives of many drivers and workers in danger. It is important to design practical and easy-to-deploy real-world systems to detect the onset of drowsiness.In this paper, we address early drowsiness detection, which can provide early alerts and offer subjects ample time to react. We present a large and public real-life dataset of 60 subjects, with video segments labeled as alert, low vigilant, or drowsy. This dataset consists of around 30 hours of video, with contents ranging from subtle signs of drowsiness to more obvious ones. We also benchmark a temporal model for our datase…
University of Sfax（斯法克斯大学）；National School of Engineers (ENIS)（国家工程师学院（ENIS））
In this paper, we try to analyze drowsiness which is a major factor in many traffic accidents due to the clear decline in the attention and recognition of danger drivers. The object of this work is to develop an automatic method to evaluate the drowsiness stage by analysis of EEG signals records. The absolute band power of the EEG signal was computed by taking the Fast Fourier Transform (FFT) of the time series signal. Finally, the algorithm developed in this work has been improved on eight samples from the Physionet sleep-EDF database.
Modified TSception for Analyzing Driver Drowsiness and Mental Workload from EEG
Department of Computer Science and Engineering, Indian Institute of Technology, Roorkee, Uttarakhand, 247667, India（计算机科学与工程系，印度理工学院，罗尔基，乌塔兰德，印度）；Department of Computer Science and Engineering, OP Jindal University, Raigarh, Chhattisgarh, 496109, India（计算机科学与工程系，OP金达尔大学，拉里加，恰特内格尔，印度）；Department of Computer Science, Electrical and Space Engineering, Luleå Tekniska Universitet, Luleå, 97187, Sweden（计算机科学、电气与航天工程系，卢勒奥技术大学，卢勒奥，瑞典）；Department of Computer Science and Engineering, Indian Institute of Technology, Dhanbad, Jharkhand, 826007, India（计算机科学与工程系，印度理工学院，丹巴德，贾坎德，印度）
University of Sfax（斯法克斯大学）；National School of Engineers (ENIS)（国家工程师学院（ENIS））
In this study, we present a hierarchical fuzzy system by evaluating the risk state for a Driver Assistance System in order to contribute in reducing the road accident's number. A key component of this system is its ability to continually detect and test the inside and outside risks in real time: The outside car risks by detecting various road moving objects; this proposed system stands on computer vision approaches. The inside risks by presenting an automatic system for drowsy driving identification or detection by evaluating EEG signals of the driver; this developed system is based on compute…
Countless traffic accidents often occur because of the inattention of the drivers. Many factors can contribute to distractions while driving, since objects or events to physiological conditions, as drowsiness and fatigue, do not allow the driver to stay attentive. The technological progress allowed the development and application of many solutions to detect the attention in real situations, promoting the interest of the scientific community in these last years. Commonly, these solutions identify the lack of attention and alert the driver, in order to help her/him to recover the attention, avoi…
Vignan Institute of Technology and Sciences（维格纳科技学院）；VITS（维格纳科技学院）
Driving support systems, such as car navigation systems are becoming common and they support driver in several aspects. Non-intrusive method of detecting Fatigue and drowsiness based on eye-blink count and eye directed instruction controlhelps the driver to prevent from collision caused by drowsy driving. Eye detection and tracking under various conditions such as illumination, background, face alignment and facial expression makes the problem complex.Neural Network based algorithm is proposed in this paper to detect the eyes efficiently. In the proposed algorithm, first the neural Network is …
Identity recognition in a car cabin is a critical task nowadays and offers a great field of applications ranging from personalizing intelligent cars to suit drivers physical and behavioral needs to increasing safety and security. However, the performance and applicability of published approaches are still not suitable for use in series cars and need to be improved. In this paper, we investigate Human Identity Recognition in a car cabin with Time Series Classification (TSC) and deep neural networks. We use gas and brake pedal pressure as input to our models. This data is easily collectable duri…
University of Tehran（德黑兰大学）；Amirkabir University of Technology（阿米尔卡比尔理工大学）；Henry Ford Health System（亨利福特医疗系统）
Heart rate and blink duration are two vital physiological signals which give information about cardiac activity and consciousness. Monitoring these two signals is crucial for various applications such as driver drowsiness detection. As there are several problems posed by the conventional systems to be used for continuous, long-term monitoring, a remote blink and ECG monitoring system can be used as an alternative. For estimating the blink duration, two strategies are used. In the first approach, pictures of open and closed eyes are fed into an Artificial Neural Network (ANN) to decide whether …
Computer Vision is considered to be one of the most important areas in research and has focused on developing many applications that has proved to be useful for both research and societal benefits. Today we have been witnessing many of the road mishaps happening just because of the lack of concentration while driving.As a part of avoiding this kind of disaster happening in day to day life there are many technologies focusing on keeping track of the vehicle drivers concentration.One such technology uses the method of eye blink detection to find out the concentration level of the driver.With the…
Xperi Corporation（Xperi公司）；National University of Ireland Galway（爱尔兰国立高威大学）；Prophesee（Prophesee公司）
Event cameras contain emerging, neuromorphic vision sensors that capture local light intensity changes at each pixel, generating a stream of asynchronous events. This way of acquiring visual information constitutes a departure from traditional frame based cameras and offers several significant advantages: low energy consumption, high temporal resolution, high dynamic range and low latency. Driver monitoring systems (DMS) are in-cabin safety systems designed to sense and understand a drivers physical and cognitive state. Event cameras are particularly suited to DMS due to their inherent advanta…
Driver vigilance estimation is an important task for transportation safety. Wearable and portable brain-computer interface devices provide a powerful means for real-time monitoring of the vigilance level of drivers to help with avoiding distracted or impaired driving. In this paper, we propose a novel multimodal architecture for in-vehicle vigilance estimation from Electroencephalogram and Electrooculogram. To enable the system to focus on the most salient parts of the learned multimodal representations, we propose an architecture composed of a capsule attention mechanism following a deep Long…
Izmir Institute of Technology（伊兹密尔理工学院）；TTTech Auto Turkey Software（TTTech汽车土耳其软件公司）
Advanced driver assistance and automated driving systems should be capable of predicting and avoiding dangerous situations. This study proposes a method to predict potentially dangerous cut-in maneuvers happening in the ego lane. We follow a computer vision-based approach that only employs a single in-vehicle RGB camera, and we classify the target vehicle's maneuver based on the recent video frames. Our algorithm consists of a CNN-based vehicle detection and tracking step and an LSTM-based maneuver classification step. It is more computationally efficient than other vision-based methods since …
University of Texas at Dallas（德克萨斯大学达拉斯分校）；Honda Research Institute（本田研究所）
Anomaly driving detection is an important problem in advanced driver assistance systems (ADAS). It is important to identify potential hazard scenarios as early as possible to avoid potential accidents. This study proposes an unsupervised method to quantify driving anomalies using a conditional generative adversarial network (GAN). The approach predicts upcoming driving scenarios by conditioning the models on the previously observed signals. The system uses the difference of the output from the discriminator between the predicted and actual signals as a metric to quantify the anomaly degree of …
