In this paper, we present a parallel architecture for a sensor fusion detection system that combines a camera and 1D light detection and ranging (lidar) sensor for object detection. The system contains two object detection methods, one based on an optical flow, and the other using lidar. The two sensors can effectively complement the defects of the other. The accurate longitudinal accuracy of the object's location and its lateral movement information can be achieved simultaneously. Using a spatio-temporal alignment and a policy of sensor fusion, we completed the development of a fusion detection system with high reliability at distances of up to 20 m. Test results show that the proposed system achieves a high level of accuracy for pedestrian or object detection in front of a vehicle, and has high robustness to special environments.
University of California, San Diego（加利福尼亚大学圣地亚哥分校）；Laboratory for Intelligent and Safe Automobiles（智能与安全汽车实验室）
The intelligent vehicle community has devoted considerable efforts to model driver behavior, and in particular to detect and overcome driver distraction in an effort to reduce accidents caused by driver negligence. However, as the domain increasingly shifts towards autonomous and semi-autonomous solutions, the driver is no longer integral to the decision making process, indicating a need to refocus efforts elsewhere. To this end, we propose to study pedestrian distraction instead. In particular, we focus on detecting pedestrians who are engaged in secondary activities involving their cellphone…
Myongji University（明知大学）；Chonnam University（全南大学）；HCMC University of Technology and Education（胡志明市技术师范大学）
A unified system integrating a compact object detector and a surrounding environmental condition classifier for enhancing the robustness of object detection scheme in advanced driver assistance systems (ADAS) is proposed in this paper. ADAS are invented to improve traffic safety and effectiveness in autonomous driving systems where object detection plays an extremely important role. However, modern object detectors integrated in ADAS are still unstable due to high latency and the variation of the environmental contexts in the deployment phase. Our system is proposed to address the aforemention…
National University of Ireland Galway（爱尔兰国立高威大学）；XPERI Corporation（XPERI公司）
Object detection in thermal infrared spectrum provides more reliable data source in low-lighting conditions and different weather conditions, as it is useful both in-cabin and outside for pedestrian, animal, and vehicular detection as well as for detecting street-signs & lighting poles. This paper is about exploring and adapting state-of-the-art object detection and classifier framework on thermal vision with seven distinct classes for advanced driver-assistance systems (ADAS). The trained network variants on public datasets are validated on test data with three different test approaches which…
Traditional pedestrian collision warning systems sometimes raise alarms even when there is no danger (e.g., when all pedestrians are walking on the sidewalk). These false alarms can make it difficult for drivers to concentrate on their driving. In this paper, we propose a novel framework for an end-to-end pedestrian collision warning system based on a convolutional neural network. Semantic segmentation information is used to train the convolutional neural network and two loss functions, such as cross entropy and Euclidean losses, are minimized. Finally, we demonstrate the effectiveness of our …
WTEFNet: Real-Time Low-Light Object Detection for Advanced Driver Assistance Systems
Guangdong Provincial Key Laboratory of Intelligent Transportation System, School of Intelligent Systems Engineering, Sun Yat-sen University（广东省智能交通系统重点实验室，智能系统工程学院，中山大学）；Intelligent Transportation Systems Research Center, Wuhan University of Technology（智能交通系统研究中心，武汉理工大学）；State Key Laboratory of Automotive Simulation and Control, Jilin University（汽车模拟与控制国家重点实验室，吉林大学）；School of Transportation, Southeast University（交通学院，东南大学）；Department of Civil and Environmental Engineering, University of Alberta（土木与环境工程系，阿尔伯塔大学）
ShrinkBox: Backdoor Attack on Object Detection to Disrupt Collision Avoidance in Machine Learning-based Advanced Driver Assistance Systems
eBRAIN Lab, New York University Abu Dhabi (NYUAD), UAE（eBRAIN实验室，纽约大学阿布扎赫尔分校（NYUAD），阿联酋）；AI and Digital Science Research Center, Technology Innovation Institute (TII), Abu Dhabi, UAE（人工智能与数字科学研究中心，技术创新研究所（TII），阿布扎赫尔，阿联酋）
Dr. Ing. h.c. F. Porsche AG（保时捷股份公司）；Robert Bosch GmbH（罗伯特·博世有限公司）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；NEC Laboratories Europe GmbH（NEC欧洲实验室有限公司）；Leibniz University Hannover（汉诺威莱布尼茨大学）
In recent years, computer vision algorithms have become more powerful. However, current algorithms mainly share one limitation: They rely on directly visible objects. This is a significant drawback compared to human behavior, where visual cues caused by objects (e.g., shadows) are already used intuitively to retrieve information or anticipate occurring objects. While driving at night, this performance deficit becomes even more obvious: Humans already process the light artifacts caused by the headlamps of oncoming vehicles to estimate where they appear, whereas current object detection systems …
Universitat Autònoma de Barcelona（巴塞罗那自治大学）；Computer Vision Center（计算机视觉中心）
Anticipating the intentions of vulnerable road users (VRUs) such as pedestrians and cyclists is critical for performing safe and comfortable driving maneuvers. This is the case for human driving and, thus, should be taken into account by systems providing any level of driving assistance, from advanced driver assistant systems (ADAS) to fully autonomous vehicles (AVs). In this paper, we show how the latest advances on monocular vision-based human pose estimation, i.e. those relying on deep Convolutional Neural Networks (CNNs), enable to recognize the intentions of such VRUs. In the case of cycl…
Vehicles are constantly approaching and sharing the road with pedestrians, and as a result it is critical for vehicles to prevent any collisions with pedestrians. Current methods for pedestrian collision prevention focus on integrating visual pedestrian detectors with Automatic Emergency Braking (AEB) systems which can trigger warnings and apply brakes as a pedestrian enters a vehicle's path. Unfortunately, pedestrian-detection-based systems can be hindered in certain situations such as nighttime or when pedestrians are occluded. Our system, CHAMP (Crowdsourced, History-based Advisories of Map…
Lacking the ability to sense ambient environments effectively, blind and visually impaired people (BVIP) face difficulty in walking outdoors, especially in urban areas. Therefore, tools for assisting BVIP are of great importance. In this paper, we propose a novel "flying guide dog" prototype for BVIP assistance using drone and street view semantic segmentation. Based on the walkable areas extracted from the segmentation prediction, the drone can adjust its movement automatically and thus lead the user to walk along the walkable path. By recognizing the color of pedestrian traffic lights, our p…
Today, one of the major challenges that autonomous vehicles are facing is the ability to drive in urban environments. Such a task requires communication between autonomous vehicles and other road users in order to resolve various traffic ambiguities. The interaction between road users is a form of negotiation in which the parties involved have to share their attention regarding a common objective or a goal (e.g. crossing an intersection), and coordinate their actions in order to accomplish it. In this literature review we aim to address the interaction problem between pedestrians and drivers (…
Vision-based driver assistance systems is one of the rapidly growing research areas of ITS, due to various factors such as the increased level of safety requirements in automotive, computational power in embedded systems, and desire to get closer to autonomous driving. It is a cross disciplinary area encompassing specialised fields like computer vision, machine learning, robotic navigation, embedded systems, automotive electronics and safety critical software. In this paper, we survey the list of vision based advanced driver assistance systems with a consistent terminology and propose a taxono…
We propose a new deep learning based framework to identify pedestrians, and caution distracted drivers, in an effort to prevent the loss of life and property. This framework uses two Convolutional Neural Networks (CNN), one which detects pedestrians and the second which predicts the onset of drowsiness in a driver, is implemented on a Raspberry Pi 3 Model B+, shows great promise. The algorithm for implementing such a low-cost, low-compute model is presented and the results discussed.
Massachusetts Institute of Technology（麻省理工学院）；Toyota Collaborative Safety Research Center（丰田协作安全研究中心）
Humans, as both pedestrians and drivers, generally skillfully navigate traffic intersections. Despite the uncertainty, danger, and the non-verbal nature of communication commonly found in these interactions, there are surprisingly few collisions considering the total number of interactions. As the role of automation technology in vehicles grows, it becomes increasingly critical to understand the relationship between pedestrian and driver behavior: how pedestrians perceive the actions of a vehicle/driver and how pedestrians make crossing decisions. The relationship between time-to-arrival (TTA)…
Computer Vision, either alone or combined with other technologies such as radar or Lidar, is one of the key technologies used in Advanced Driver Assistance Systems (ADAS). Its role understanding and analysing the driving scene is of great importance as it can be noted by the number of ADAS applications that use this technology. However, porting a vision algorithm to an embedded automotive system is still very challenging, as there must be a trade-off between several design requisites. Furthermore, there is not a standard implementation platform, so different alternatives have been proposed by …
This paper presents the experimental comparison of fourteen stereo matching algorithms in variant illumination conditions. Different adaptations of global and local stereo matching techniques are chosen for evaluation The variant strength and weakness of the chosen correspondence algorithms are explored by employing the methodology of the prediction error strategy. The algorithms are gauged on the basis of their performance on real world data set taken in various indoor lighting conditions and at different times of the day
Lane Detection System for Driver Assistance in Vehicles
Institute of Informatics Federal University of Goiás Goiânia, Goiás；School of Electrical, Mechanical；Computer Engineering Federal University of Goiás Goiânia, Goiás；Institute of Informatics Federal University of Goiás Goiânia, Goiás matheus\
This paper proposes a concept of computer vision-based guidance assistance for agricultural vehicles to increase the accuracy in plowing and reduce driver's cognitive burden in long-lasting tillage operations. Plowing is a common agricultural practice to prepare the soil for planting in many countries and it can take place both in the spring and the fall. Since plowing operation requires high traction forces, it causes increased energy consumption. Moreover, longer operation time due to unnecessary maneuvers leads to higher fuel consumption. To provide necessary information for the driver and …
A complete overview of the surrounding vehicle environment is important for driver assistance systems and highly autonomous driving. Fusing results of multiple sensor types like camera, radar and lidar is crucial for increasing the robustness. The detection and classification of objects like cars, bicycles or pedestrians has been analyzed in the past for many sensor types. Beyond that, it is also helpful to refine these classes and distinguish for example between different pedestrian types or activities. This task is usually performed on camera data, though recent developments are based on rad…
Human-computer interaction (HCI) is crucial for the safety of lives as autonomous vehicles (AVs) become commonplace. Yet, little effort has been put toward ensuring that AVs understand humans on the road. In this paper, we present GLADAS, a simulator-based research platform designed to teach AVs to understand pedestrian hand gestures. GLADAS supports the training, testing, and validation of deep learning-based self-driving car gesture recognition systems. We focus on gestures as they are a primordial (i.e, natural and common) way to interact with cars. To the best of our knowledge, GLADAS is t…
With the gradual application of infrared night vision vehicle assistance system in automatic driving, the accuracy of the collected infrared images of pedestrians is gradually improved. In this paper, the migration learning method is used to apply YOLOv3 model to realize pedestrian target detection in infrared images. The target detection model YOLOv3 is migrated to the CVC infrared pedestrian data set, and Diou loss is used to replace the loss function of the original YOLO model to test different super parameters to obtain the best migration learning effect. The experimental results show that…
National University of Ireland, Galway（爱尔兰国立大学戈尔韦）；Atlantic Technological University（大西洋理工大学）
Pedestrian detection is among the most safety-critical features of driver assistance systems for autonomous vehicles. One of the most complex detection challenges is that of partial occlusion, where a target object is only partially available to the sensor due to obstruction by another foreground object. A number of current pedestrian detection benchmarks provide annotation for partial occlusion to assess algorithm performance in these scenarios, however each benchmark varies greatly in their definition of the occurrence and severity of occlusion. In addition, current occlusion level annotatio…
University of Guilan（吉兰大学）；University of Luxembourg（卢森堡大学）
Nowadays, utilizing Advanced Driver-Assistance Systems (ADAS) has absorbed a huge interest as a potential solution for reducing road traffic issues. Despite recent technological advances in such systems, there are still many inquiries that need to be overcome. For instance, ADAS requires accurate and real-time detection of pedestrians in various driving scenarios. To solve the mentioned problem, this paper aims to fine-tune the YOLOv5s framework for handling pedestrian detection challenges on the real-world instances of Caltech pedestrian dataset. We also introduce a developed toolbox for prep…
Studies of object detection and localization, particularly pedestrian detection have received considerable attention in recent times due to its several prospective applications such as surveillance, driving assistance, autonomous cars, etc. Also, a significant trend of latest research studies in related problem areas is the use of sophisticated Deep Learning based approaches to improve the benchmark performance on various standard datasets. A trade-off between the speed (number of video frames processed per second) and detection accuracy has often been reported in the existing literature. In t…
University of Sfax（斯法克斯大学）；National School of Engineers (ENIS)（国家工程师学院（ENIS））
In this study, we present a hierarchical fuzzy system by evaluating the risk state for a Driver Assistance System in order to contribute in reducing the road accident's number. A key component of this system is its ability to continually detect and test the inside and outside risks in real time: The outside car risks by detecting various road moving objects; this proposed system stands on computer vision approaches. The inside risks by presenting an automatic system for drowsy driving identification or detection by evaluating EEG signals of the driver; this developed system is based on compute…
Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）；Beamagine S.L.（Beamagine公司）
Detecting pedestrians is a crucial task in autonomous driving systems to ensure the safety of drivers and pedestrians. The technologies involved in these algorithms must be precise and reliable, regardless of environment conditions. Relying solely on RGB cameras may not be enough to recognize road environments in situations where cameras cannot capture scenes properly. Some approaches aim to compensate for these limitations by combining RGB cameras with TOF sensors, such as LIDARs. However, there are few works that address this problem using exclusively the 3D geometric information provided by…
Accurate and efficient pedestrian detection is crucial for the intelligent transportation system regarding pedestrian safety and mobility, e.g., Advanced Driver Assistance Systems, and smart pedestrian crosswalk systems. Among all pedestrian detection methods, vision-based detection method is demonstrated to be the most effective in previous studies. However, the existing vision-based pedestrian detection algorithms still have two limitations that restrict their implementations, those being real-time performance as well as the resistance to the impacts of environmental factors, e.g., low illum…
Predicting pedestrian crossing intention is an indispensable aspect of deploying advanced driving systems (ADS) or advanced driver-assistance systems (ADAS) to real life. State-of-the-art methods in predicting pedestrian crossing intention often rely on multiple streams of information as inputs, each of which requires massive computational resources and heavy network architectures to generate. However, such reliance limits the practical application of the systems. In this paper, driven the the real-world demands of pedestrian crossing intention prediction models with both high efficiency and a…
Internet Interdisciplinary Institute (IN3), Universitat Oberta de Catalunya（互联网跨学科研究所（IN3），加泰罗尼亚开放大学）
Increased interaction between and among pedestrians and vehicles in the crowded urban environments of today gives rise to a negative side-effect: a growth in traffic accidents, with pedestrians being the most vulnerable elements. Recent work has shown that Convolutional Neural Networks are able to accurately predict accident rates exploiting Street View imagery along urban roads. The promising results point to the plausibility of aided design of safe urban landscapes, for both pedestrians and vehicles. In this paper, by considering historical accident data and Street View images, we detail how…
Massachusetts Institute of Technology (MIT)（麻省理工学院（MIT））
Automated estimation of the allocation of a driver's visual attention may be a critical component of future Advanced Driver Assistance Systems. In theory, vision-based tracking of the eye can provide a good estimate of gaze location. In practice, eye tracking from video is challenging because of sunglasses, eyeglass reflections, lighting conditions, occlusions, motion blur, and other factors. Estimation of head pose, on the other hand, is robust to many of these effects, but cannot provide as fine-grained of a resolution in localizing the gaze. However, for the purpose of keeping the driver sa…
Chalmers University of Technology（查尔姆斯理工大学）；Volvo Cars Technology USA（沃尔沃汽车美国技术公司）；Volvo Car Corporation（沃尔沃汽车公司）；University of California, Berkeley（加州大学伯克利分校）
Pedestrian intention recognition is very important to develop robust and safe autonomous driving (AD) and advanced driver assistance systems (ADAS) functionalities for urban driving. In this work, we develop an end-to-end pedestrian intention framework that performs well on day- and night- time scenarios. Our framework relies on objection detection bounding boxes combined with skeletal features of human pose. We study early, late, and combined (early and late) fusion mechanisms to exploit the skeletal features and reduce false positives as well to improve the intention prediction performance. …
