With the rapid development of technology, automobiles have become an essential asset in our day-to-day lives. One of the more important researches is Traffic Signs Recognition (TSR) systems. This paper describes an approach for efficiently detecting and recognizing traffic signs in real-time, taking into account the various weather, illumination and visibility challenges through the means of transfer learning. We tackle the traffic sign detection problem using the state-of-the-art of multi-object detection systems such as Faster Recurrent Convolutional Neural Networks (F-RCNN) and Single Shot Multi- Box Detector (SSD) combined with various feature extractors such as MobileNet v1 and Inception v2, and also Tiny-YOLOv2. However, the focus of this paper is going to be F-RCNN Inception v2 and Tiny YOLO v2 as they achieved the best results. The aforementioned models were fine-tuned on the German Traffic Signs Detection Benchmark (GTSDB) dataset. These models were tested on the host PC as well as Raspberry Pi 3 Model B+ and the TASS PreScan simulation. We will discuss the results of all the models in the conclusion section.
Military Institute of Science & Technology (MIST)（军事科学技术研究院）
Computer vision coupled with Deep Learning (DL) techniques bring out a substantial prospect in the field of traffic control, monitoring and law enforcing activities. This paper presents a YOLOv4 object detection model in which the Convolutional Neural Network (CNN) is trained and tuned for detecting the license plate of the vehicles of Bangladesh and recognizing characters using tesseract from the detected license plates. Here we also present a Graphical User Interface (GUI) based on Tkinter, a python package. The license plate detection model is trained with mean average precision (mAP) of 90…
Contrastive Learning-Driven Traffic Sign Perception: Multi-Modal Fusion of Text and Vision
School of Intelligent Systems Engineering, Sun Yat-sen University（中山大学智能系统工程学院）；Guangdong Provincial Key Laboratory of Intelligent Transportation System（广东省智能交通系统重点实验室）
Shenzhen Kaiyuan Internet Security Co., Ltd（深圳开元互联网安全有限公司）；University of Maryland（马里兰大学）
Adversarial Examples (AEs) can deceive Deep Neural Networks (DNNs) and have received a lot of attention recently. However, majority of the research on AEs is in the digital domain and the adversarial patches are static, which is very different from many real-world DNN applications such as Traffic Sign Recognition (TSR) systems in autonomous vehicles. In TSR systems, object detectors use DNNs to process streaming video in real time. From the view of object detectors, the traffic sign`s position and quality of the video are continuously changing, rendering the digital AEs ineffective in the phys…
Smart automated traffic enforcement solutions have been gaining popularity in recent years. These solutions are ubiquitously used for seat-belt violation detection, red-light violation detection and speed violation detection purposes. Highly accurate license plate recognition is an indispensable part of these systems. However, general license plate recognition systems require high resolution images for high performance. In this study, we propose a novel license plate recognition method for general roadway surveillance cameras. Proposed segmentation free license plate recognition algorithm util…
This study developed a traffic sign detection and recognition algorithm based on the RetinaNet. Two main aspects were revised to improve the detection of traffic signs: image cropping to address the issue of large image and small traffic signs; and using more anchors with various scales to detect traffic signs with different sizes and shapes. The proposed algorithm was trained and tested in a series of autonomous driving front-view images in a virtual simulation environment. Results show that the algorithm performed extremely well under good illumination and weather conditions. Its drawbacks a…
Al-Ahliyya Amman University（安曼阿赫利亚大学）；Budapest University of Technology and Economics（布达佩斯技术与经济大学）；Queensland University of Technology（昆士兰科技大学）；Arab American University（阿拉伯美国大学）；Columbia University（哥伦比亚大学）
This paper presents a method for detection and recognition of traffic signs based on information extracted from an event camera. The solution used a FireNet deep convolutional neural network to reconstruct events into greyscale frames. Two YOLOv4 network models were trained, one based on greyscale images and the other on colour images. The best result was achieved for the model trained on the basis of greyscale images, achieving an efficiency of 87.03%.
Gwangju Institute of Science and Technology (GIST)（光州科学技术院）；Curtin University（科廷大学）
Traffic scene recognition, which requires various visual classification tasks, is a critical ingredient in autonomous vehicles. However, most existing approaches treat each relevant task independently from one another, never considering the entire system as a whole. Because of this, they are limited to utilizing a task-specific set of features for all possible tasks of inference-time, which ignores the capability to leverage common task-invariant contextual knowledge for the task at hand. To address this problem, we propose an algorithm to jointly learn the task-specific and shared representat…
Traffic light recognition, as a critical component of the perception module of self-driving vehicles, plays a vital role in the intelligent transportation systems. The prevalent deep learning based traffic light recognition methods heavily hinge on the large quantity and rich diversity of training data. However, it is quite challenging to collect data in various rare scenarios such as flashing, blackout or extreme weather, thus resulting in the imbalanced distribution of training data and consequently the degraded performance in recognizing rare classes. In this paper, we seek to improve traff…
School of Automation, Southeast University（东南大学自动化学院）
Researches show that fatigue driving is one of the important causes of road traffic accidents, so it is of great significance to study the driver fatigue recognition algorithm to improve road traffic safety. In recent years, with the development of deep learning, the field of pattern recognition has made great development. This paper designs a real-time fatigue state recognition algorithm based on spatio-temporal feature sequence, which can be mainly applied to the scene of fatigue driving recognition. The algorithm is divided into three task networks: face detection network, facial landmark d…
Faculty of Computer and Information Science, University of Ljubljana（卢布尔雅那大学计算机与信息科学学院）
Automatic detection and recognition of traffic signs plays a crucial role in management of the traffic-sign inventory. It provides accurate and timely way to manage traffic-sign inventory with a minimal human effort. In the computer vision community the recognition and detection of traffic signs is a well-researched problem. A vast majority of existing approaches perform well on traffic signs needed for advanced drivers-assistance and autonomous systems. However, this represents a relatively small number of all traffic signs (around 50 categories out of several hundred) and performance on the …
University Of Technology of Compiegne（贡比涅技术大学）；Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）；Zenseact
We describe an iterative active-learning algorithm to recognise rare traffic signs. A standard ResNet is trained on a training set containing only a single sample of the rare class. We demonstrate that by sorting the samples of a large, unlabeled set by the estimated probability of belonging to the rare class, we can efficiently identify samples from the rare class. This works despite the fact that this estimated probability is usually quite low. A reliable active-learning loop is obtained by labeling these candidate samples, including them in the training set, and iterating the procedure. Fur…
Universidade Federal do Espírito Santo（圣灵联邦大学）；Instituto Federal do Espírito Santo（圣灵联邦研究所）
Autonomous terrestrial vehicles must be capable of perceiving traffic lights and recognizing their current states to share the streets with human drivers. Most of the time, human drivers can easily identify the relevant traffic lights. To deal with this issue, a common solution for autonomous cars is to integrate recognition with prior maps. However, additional solution is required for the detection and recognition of the traffic light. Deep learning techniques have showed great performance and power of generalization including traffic related problems. Motivated by the advances in deep learni…
A car driver knows how to react on the gestures of the traffic officers. Clearly, this is not the case for the autonomous vehicle, unless it has road traffic control gesture recognition functionalities. In this work, we address the limitation of the existing autonomous driving datasets to provide learning data for traffic control gesture recognition. We introduce a dataset that is based on 3D body skeleton input to perform traffic control gesture classification on every time step. Our dataset consists of 250 sequences from several actors, ranging from 16 to 90 seconds per sequence. To evaluate…
Beijing University of Technology（北京工业大学）；University of British Columbia（不列颠哥伦比亚大学）；Peking University（北京大学）
The paper presents a Traffic Sign Recognition (TSR) system, which can fast and accurately recognize traffic signs of different sizes in images. The system consists of two well-designed Convolutional Neural Networks (CNNs), one for region proposals of traffic signs and one for classification of each region. In the proposal CNN, a Fully Convolutional Network (FCN) with a dual multi-scale architecture is proposed to achieve scale invariant detection. In training the proposal network, a modified "Online Hard Example Mining" (OHEM) scheme is adopted to suppress false positives. The classification n…
TSCLIP: Robust CLIP Fine-Tuning for Worldwide Cross-Regional Traffic Sign Recognition
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Wuhan Polytechnic University（武汉轻工大学）；The Hong Kong University of Science and Technology（香港科技大学）
Traffic scene recognition is an important and challenging issue in Intelligent Transportation Systems (ITS). Recently, Convolutional Neural Network (CNN) models have achieved great success in many applications, including scene classification. The remarkable representational learning capability of CNN remains to be further explored for solving real-world problems. Vector of Locally Aggregated Descriptors (VLAD) encoding has also proved to be a powerful method in catching global contextual information. In this paper, we attempted to solve the traffic scene recognition problem by combining the fe…
Hochschule Bonn-Rhein-Sieg（波恩-莱茵-锡格应用技术大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
Traffic sign recognition is an important component of many advanced driving assistance systems, and it is required for full autonomous driving. Computational performance is usually the bottleneck in using large scale neural networks for this purpose. SqueezeNet is a good candidate for efficient image classification of traffic signs, but in our experiments it does not reach high accuracy, and we believe this is due to lack of data, requiring data augmentation. Generative adversarial networks can learn the high dimensional distribution of empirical data, allowing the generation of new data point…
Recent advances in visual recognition show overarching success by virtue of large amounts of supervised data. However,the acquisition of a large supervised dataset is often challenging. This is also true for intelligent transportation applications, i.e., traffic sign recognition. For example, a model trained with data of one country may not be easily generalized to another country without much data. We propose a novel feature embedding scheme for unseen class classification when the representative class template is given. Traffic signs, unlike other objects, have official images. We perform co…
In this paper, we investigate the robustness of traffic sign recognition algorithms under challenging conditions. Existing datasets are limited in terms of their size and challenging condition coverage, which motivated us to generate the Challenging Unreal and Real Environments for Traffic Sign Recognition (CURE-TSR) dataset. It includes more than two million traffic sign images that are based on real-world and simulator data. We benchmark the performance of existing solutions in real-world scenarios and analyze the performance variation with respect to challenging conditions. We show that cha…
Traffic Incident Database with Multiple Labels Including Various Perspective Environmental Information
Aichi Institute of Technology（爱知工业大学）；Tokyo Denki University（东京电机大学）；National Institute of Advanced Industrial Science and Technology (AIST)（日本产业技术综合研究所（AIST））；Fukuoka University（福冈大学）；Keio University（庆应义塾大学）
Traffic signs support road safety and managing the flow of traffic, hence are an integral part of any vision system for autonomous driving. While the use of deep learning is well-known in traffic signs classification due to the high accuracy results obtained using convolutional neural networks (CNNs) (state of the art is 99.46\%), little is known about binarized neural networks (BNNs). Compared to CNNs, BNNs reduce the model size and simplify convolution operations and have shown promising results in computationally limited and energy-constrained devices which appear in the context of autonomo…
University of Waterloo（滑铁卢大学）；DarwinAI Corp.（DarwinAI公司）
Traffic sign recognition is a very important computer vision task for a number of real-world applications such as intelligent transportation surveillance and analysis. While deep neural networks have been demonstrated in recent years to provide state-of-the-art performance traffic sign recognition, a key challenge for enabling the widespread deployment of deep neural networks for embedded traffic sign recognition is the high computational and memory requirements of such networks. As a consequence, there are significant benefits in investigating compact deep neural network architectures for tra…
We propose a traffic danger recognition model that works with arbitrary traffic surveillance cameras to identify and predict car crashes. There are too many cameras to monitor manually. Therefore, we developed a model to predict and identify car crashes from surveillance cameras based on a 3D reconstruction of the road plane and prediction of trajectories. For normal traffic, it supports real-time proactive safety checks of speeds and distances between vehicles to provide insights about possible high-risk areas. We achieve good prediction and recognition of car crashes without using any labele…
School of Information Technology, Halmstad University（哈姆斯塔德大学信息技术学院）；RISE Viktoria
Action and intention recognition of pedestrians in urban settings are challenging problems for Advanced Driver Assistance Systems as well as future autonomous vehicles to maintain smooth and safe traffic. This work investigates a number of feature extraction methods in combination with several machine learning algorithms to build knowledge on how to automatically detect the action and intention of pedestrians in urban traffic. We focus on the motion and head orientation to predict whether the pedestrian is about to cross the street or not. The work is based on the Joint Attention for Autonomou…
Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）；The University of Chicago（芝加哥大学）
Traffic light recognition is essential for fully autonomous driving in urban areas. In this paper, we investigate the feasibility of fooling traffic light recognition mechanisms by shedding laser interference on the camera. By exploiting the rolling shutter of CMOS sensors, we manage to inject a color stripe overlapped on the traffic light in the image, which can cause a red light to be recognized as a green light or vice versa. To increase the success rate, we design an optimization method to search for effective laser parameters based on empirical models of laser interference. Our evaluation…
Currently, the visually impaired rely on either a sighted human, guide dog, or white cane to safely navigate. However, the training of guide dogs is extremely expensive, and canes cannot provide essential information regarding the color of traffic lights and direction of crosswalks. In this paper, we propose a deep learning based solution that provides information regarding the traffic light mode and the position of the zebra crossing. Previous solutions that utilize machine learning only provide one piece of information and are mostly binary: only detecting red or green lights. The proposed c…
Valeo Driving Assistance Domain（法雷奥驾驶辅助领域）；Ecole des Mines de Paris (ParisTech)（巴黎矿业学院（巴黎理工学院））
We present a new modular traffic signs recognition system, successfully applied to both American and European speed limit signs. Our sign detection step is based only on shape-detection (rectangles or circles). This enables it to work on grayscale images, contrary to most European competitors, which eases robustness to illumination conditions (notably night operation). Speed sign candidates are classified (or rejected) by segmenting potential digits inside them (which is rather original and has several advantages), and then applying a neural digit recognition. The global detection rate is ~90%…
Traffic Sign Recognition in Autonomous Driving: Dataset, Benchmark, and Field Experiment
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Lingnan University（岭大）；Shenzhen Unity Drive Innovation Technology Co., Ltd.（深圳Unity Drive创新技术有限公司）；The Hong Kong University of Science and Technology（香港科技大学）
Enhancing LLM-based Autonomous Driving with Modular Traffic Light and Sign Recognition
Institute for Intelligent Systems（智能系统研究所）；Esslingen University of Applied Sciences（应用科学大学埃斯林根）；Department of Computer Science（计算机科学系）；University of Freiburg（弗赖堡大学）
Shah and Anchor Kutchhi Engineering College（沙阿与安克尔库奇工程学院）
Vehicle tracking is an integral part of intelligent traffic management systems. Previous implementations of vehicle tracking used Global Positioning System(GPS) based systems that gave location of the vehicle of an individual on their smartphones.The proposed system uses a novel approach to vehicle tracking using Vehicle License plate detection and recognition (VLPR) technique, which can be integrated on a large scale with traffic management systems. Initial methods of implementing VLPR used simple image processing techniques which were quite experimental and heuristic. With the onset of Deep …
Brno University of Technology（布尔诺理工大学）；Faculty of Information Technology（信息技术学院）；Centre of Excellence IT4Innovations（IT4创新卓越中心）
In this paper, we focus on fine-grained recognition of vehicles mainly in traffic surveillance applications. We propose an approach that is orthogonal to recent advancements in fine-grained recognition (automatic part discovery and bilinear pooling). In addition, in contrast to other methods focused on fine-grained recognition of vehicles, we do not limit ourselves to a frontal/rear viewpoint, but allow the vehicles to be seen from any viewpoint. Our approach is based on 3-D bounding boxes built around the vehicles. The bounding box can be automatically constructed from traffic surveillance da…
Skolkovo Institute of Science and Technology (Skoltech)（斯科尔科沃科学技术学院（Skoltech））；NtechLab（NtechLab公司）；Samsung AI Center, Moscow（三星莫斯科人工智能中心）；Reshetnev Siberian State University of Science and Technology（列舍特涅夫西伯利亚国立科技大学）
With the advancements of various autonomous car projects aiming to achieve SAE Level 5, real-time detection of traffic signs in real-life scenarios has become a highly relevant problem for the industry. Even though a great progress has been achieved in this field, there is still no clear consensus on what the state-of-the-art in this field is. Moreover, it is important to develop and test systems in various regions and conditions. This is why the "Ice Vision" competition has focused on the detection of Russian traffic signs in winter conditions. The IceVisionSet dataset used for this competiti…
Deep Neural Networks (DNNs) are increasingly applied in the real world in safety critical applications like advanced driver assistance systems. An example for such use case is represented by traffic sign recognition systems. At the same time, it is known that current DNNs can be fooled by adversarial attacks, which raises safety concerns if those attacks can be applied under realistic conditions. In this work we apply different black-box attack methods to generate perturbations that are applied in the physical environment and can be used to fool systems under different environmental conditions…
