We propose a method for automatic calibration of a traffic surveillance camera with wide-angle lenses. Video footage of a few minutes is sufficient for the entire calibration process to take place. This method takes in the height of the camera from the ground plane as the only user input to overcome the scale ambiguity. The calibration is performed in two stages, 1. Intrinsic Calibration 2. Extrinsic Calibration. Intrinsic calibration is achieved by assuming an equidistant fisheye distortion and an ideal camera model. Extrinsic calibration is accomplished by estimating the two vanishing points, on the ground plane, from the motion of vehicles at perpendicular intersections. The first stage of intrinsic calibration is also valid for thermal cameras. Experiments have been conducted to demonstrate the effectiveness of this approach on visible as well as thermal cameras.
Index Terms: fish-eye, calibration, thermal camera, intelligent transportation systems, vanishing points
We focus on the problem of detecting traffic events in a surveillance scenario, including the detection of both vehicle actions and traffic collisions. Existing event detection systems are mostly learning-based and have achieved convincing performance when a large amount of training data is available. However, in real-world scenarios, collecting sufficient labeled training data is expensive and sometimes impossible (e.g. for traffic collision detection). Moreover, the conventional 2D representation of surveillance views is easily affected by occlusions and different camera views in nature. To …
AI Innovation and Application Center of China Unicom（中国联通人工智能创新应用中心）
Automatic traffic accidents detection has appealed to the machine vision community due to its implications on the development of autonomous intelligent transportation systems (ITS) and importance to traffic safety. Most previous studies on efficient analysis and prediction of traffic accidents, however, have used small-scale datasets with limited coverage, which limits their effect and applicability. Existing datasets in traffic accidents are either small-scale, not from surveillance cameras, not open-sourced, or not built for freeway scenes. Since accidents happened in freeways tend to cause …
National Institute of Technology Warangal（瓦朗加尔国家理工学院）
Computer vision-based accident detection through video surveillance has become a beneficial but daunting task. In this paper, a neoteric framework for detection of road accidents is proposed. The proposed framework capitalizes on Mask R-CNN for accurate object detection followed by an efficient centroid based object tracking algorithm for surveillance footage. The probability of an accident is determined based on speed and trajectory anomalies in a vehicle after an overlap with other vehicles. The proposed framework provides a robust method to achieve a high Detection Rate and a low False Alar…
Urban Traffic Surveillance (UTS) is a surveillance system based on a monocular and calibrated video camera that detects vehicles in an urban traffic scenario with dense traffic on multiple lanes and vehicles performing sharp turning maneuvers. UTS then tracks the vehicles using a 3D bounding box representation and a physically reasonable 3D motion model relying on an unscented Kalman filter based approach. Since UTS recovers positions, shape and motion information in a three-dimensional world coordinate system, it can be employed to recognize diverse traffic violations or to supply intelligent…
Anomaly detection through video analysis is of great importance to detect any anomalous vehicle/human behavior at a traffic intersection. While most existing works use neural networks and conventional machine learning methods based on provided dataset, we will use object recognition (Faster R-CNN) to identify objects labels and their corresponding location in the video scene as the first step to implement anomaly detection. Then, the optical flow will be utilized to identify adaptive traffic flows in each region of the frame. Basically, we propose an alternative method for unusual activity det…
We propose a traffic danger recognition model that works with arbitrary traffic surveillance cameras to identify and predict car crashes. There are too many cameras to monitor manually. Therefore, we developed a model to predict and identify car crashes from surveillance cameras based on a 3D reconstruction of the road plane and prediction of trajectories. For normal traffic, it supports real-time proactive safety checks of speeds and distances between vehicles to provide insights about possible high-risk areas. We achieve good prediction and recognition of car crashes without using any labele…
Military Institute of Science & Technology (MIST)（军事科学技术研究院）
Computer vision coupled with Deep Learning (DL) techniques bring out a substantial prospect in the field of traffic control, monitoring and law enforcing activities. This paper presents a YOLOv4 object detection model in which the Convolutional Neural Network (CNN) is trained and tuned for detecting the license plate of the vehicles of Bangladesh and recognizing characters using tesseract from the detected license plates. Here we also present a Graphical User Interface (GUI) based on Tkinter, a python package. The license plate detection model is trained with mean average precision (mAP) of 90…
In this paper, we focus on fully automatic traffic surveillance camera calibration, which we use for speed measurement of passing vehicles. We improve over a recent state-of-the-art camera calibration method for traffic surveillance based on two detected vanishing points. More importantly, we propose a novel automatic scene scale inference method. The method is based on matching bounding boxes of rendered 3D models of vehicles with detected bounding boxes in the image. The proposed method can be used from arbitrary viewpoints, since it has no constraints on camera placement. We evaluate our me…
Wavelet-Enhanced Desnowing: A Novel Single Image Restoration Approach for Traffic Surveillance under Adverse Weather Conditions
Automatic Control Research Institute, China Nuclear Power Engineering Co., Ltd.（中国核电工程有限公司自动控制研究院）；CNNC Engineering Research Center for Fuel Reprocessing（中核集团燃料后处理工程研究中心）；Chinese Aeronautical Establishment, Aviation Industry Corporation of China, Ltd（中国航空工业集团有限公司中国航空研究院）
New Jersey Institute of Technology（新泽西理工学院）；Innovative AI Technologies（创新人工智能技术公司）
Automatic detection of traffic accidents is an important emerging topic in traffic monitoring systems. Nowadays many urban intersections are equipped with surveillance cameras connected to traffic management systems. Therefore, computer vision techniques can be viable tools for automatic accident detection. This paper presents a new efficient framework for accident detection at intersections for traffic surveillance applications. The proposed framework consists of three hierarchical steps, including efficient and accurate object detection based on the state-of-the-art YOLOv4 method, object tra…
Traffic control using intelligent timing of traffic lights with reinforcement learning technique and real-time processing of surveillance camera images
This paper presents the learned techniques during the Video Analysis Module of the Master in Computer Vision from the Universitat Autònoma de Barcelona, used to solve the third track of the AI-City Challenge. This challenge aims to track vehicles across multiple cameras placed in multiple intersections spread out over a city. The methodology followed focuses first in solving multi-tracking in a single camera and then extending it to multiple cameras. The qualitative results of the implemented techniques are presented using standard metrics for video analysis such as mAP for object detection an…
Indian Institute of Technology Bhubaneswar（布巴内斯瓦尔印度理工学院）
Computer vision has evolved in the last decade as a key technology for numerous applications replacing human supervision. In this paper, we present a survey on relevant visual surveillance related researches for anomaly detection in public places, focusing primarily on roads. Firstly, we revisit the surveys done in the last 10 years in this field. Since the underlying building block of a typical anomaly detection is learning, we emphasize more on learning methods applied on video scenes. We then summarize the important contributions made during last six years on anomaly detection primarily foc…
Unmanned aerial vehicles (UAVs) with mounted cameras have the advantage of capturing aerial (bird-view) images. The availability of aerial visual data and the recent advances in object detection algorithms led the computer vision community to focus on object detection tasks on aerial images. As a result of this, several aerial datasets have been introduced, including visual data with object annotations. UAVs are used solely as flying-cameras in these datasets, discarding different data types regarding the flight (e.g., time, location, internal sensors). In this work, we propose a multi-purpose…
The detection of contextual anomalies is a challenging task for surveillance since an observation can be considered anomalous or normal in a specific environmental context. An unmanned aerial vehicle (UAV) can utilize its aerial monitoring capability and employ multiple sensors to gather contextual information about the environment and perform contextual anomaly detection. In this work, we introduce a deep neural network-based method (CADNet) to find point anomalies (i.e., single instance anomalous data) and contextual anomalies (i.e., context-specific abnormality) in an environment using a UA…
Brno University of Technology（布尔诺理工大学）；Faculty of Information Technology（信息技术学院）；Centre of Excellence IT4Innovations（IT4创新卓越中心）
In this paper, we focus on fine-grained recognition of vehicles mainly in traffic surveillance applications. We propose an approach that is orthogonal to recent advancements in fine-grained recognition (automatic part discovery and bilinear pooling). In addition, in contrast to other methods focused on fine-grained recognition of vehicles, we do not limit ourselves to a frontal/rear viewpoint, but allow the vehicles to be seen from any viewpoint. Our approach is based on 3-D bounding boxes built around the vehicles. The bounding box can be automatically constructed from traffic surveillance da…
Indian Institute of Technology Bhubaneswar（印度理工学院布巴内斯瓦尔分校）；Indian Institute of Technology, Roorkee（印度理工学院鲁尔基分校）
Optimized scene representation is an important characteristic of a framework for detecting abnormalities on live videos. One of the challenges for detecting abnormalities in live videos is real-time detection of objects in a non-parametric way. Another challenge is to efficiently represent the state of objects temporally across frames. In this paper, a Gibbs sampling based heuristic model referred to as Temporal Unknown Incremental Clustering (TUIC) has been proposed to cluster pixels with motion. Pixel motion is first detected using optical flow and a Bayesian algorithm has been applied to as…
TrafficLoc: Localizing Traffic Surveillance Cameras in 3D Scenes
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；DeepScenario；Visual Geometry Group, University of Oxford（牛津大学视觉几何组）
Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校）
Predicting on-road abnormalities such as road accidents or traffic violations is a challenging task in traffic surveillance. If such predictions can be done in advance, many damages can be controlled. Here in our wok, we tried to formulate a solution for automated collision prediction in traffic surveillance videos with computer vision and deep networks. It involves object detection, tracking, trajectory estimation, and collision prediction. We propose an end-to-end collision prediction system, named as COLLIDE-PRED, that intelligently integrates the information of past and future trajectories…
Wuhan University of Technology（武汉理工大学）；National Engineering Research Center for Water Transport Safety（国家水运安全工程技术研究中心）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
The automatic identification system (AIS) and video cameras have been widely exploited for vessel traffic surveillance in inland waterways. The AIS data could provide the vessel identity and dynamic information on vessel position and movements. In contrast, the video data could describe the visual appearances of moving vessels, but without knowing the information on identity, position and movements, etc. To further improve vessel traffic surveillance, it becomes necessary to fuse the AIS and video data to simultaneously capture the visual features, identity and dynamic information for the vess…
Universidad Nacional de San Antonio Abad del Cusco（库斯科圣安东尼奥阿巴德国立大学）；Pontificia Universidad Católica del Perú（秘鲁天主教大学）
The development of autonomous vehicles arises new challenges in urban traffic scenarios where vehicle-pedestrian interactions are frequent e.g. vehicle yields to pedestrians, pedestrian slows down due approaching to the vehicle. Over the last years, several datasets have been developed to model these interactions. However, available datasets do not cover near-accident scenarios that our dataset covers. We introduce I see you, a new vehicle-pedestrian interaction dataset that tackles the lack of trajectory data in near-accident scenarios using YOLOv5 and camera calibration methods. I see you co…
Chung-Ang University（中央大学）；Department of Urban Engineering（城市工程系）
As a result of significant advances in deep learning, computer vision technology has been widely adopted in the field of traffic surveillance. Nonetheless, it is difficult to find a universal model that can measure traffic parameters irrespective of ambient conditions such as times of the day, weather, or shadows. These conditions vary recurrently, but the exact points of change are inconsistent and unpredictable. Thus, the application of a multi-regime method would be problematic, even when separate sets of model parameters are prepared in advance. In the present study we devised a robust app…
Cameras for traffic surveillance are usually pole-mounted and produce images that reflect a birds-eye view. Vehicles in such images, in general, assume an ellipse form. A bounding box for the vehicles usually includes a large empty space when the vehicle orientation is not parallel to the edges of the box. To circumvent this problem, the present study applied bounding ellipses to a non-anchor-based, single-shot detection model (CenterNet). Since this model does not depend on anchor boxes, non-max suppression (NMS) that requires computing the intersection over union (IOU) between predicted boun…
National Institute of Technology, Warangal（瓦朗加尔国家理工学院）
Vehicle theft is arguably one of the fastest-growing types of crime in India. In some of the urban areas, vehicle theft cases are believed to be around 100 each day. Identification of stolen vehicles in such precarious scenarios is not possible using traditional methods like manual checking and radio frequency identification(RFID) based technologies. This paper presents a deep learning based automatic traffic surveillance system for the detection of stolen/suspicious cars from the closed circuit television(CCTV) camera footage. It mainly comprises of four parts: Select-Detector, Image Quality …
Varying weather conditions, including rainfall and snowfall, are generally regarded as a challenge for computer vision algorithms. One proposed solution to the challenges induced by rain and snowfall is to artificially remove the rain from images or video using rain removal algorithms. It is the promise of these algorithms that the rain-removed image frames will improve the performance of subsequent segmentation and tracking algorithms. However, rain removal algorithms are typically evaluated on their ability to remove synthetic rain on a small subset of images. Currently, their behavior is un…
University of South Carolina（南卡罗来纳大学）；Cleveland State University（克利夫兰州立大学）；Alibaba Group（阿里巴巴集团）；Nanyang Technological University（南洋理工大学）
In recent years, image and video surveillance have made considerable progresses to the Intelligent Transportation Systems (ITS) with the help of deep Convolutional Neural Networks (CNNs). As one of the state-of-the-art perception approaches, detecting the interested objects in each frame of video surveillance is widely desired by ITS. Currently, object detection shows remarkable efficiency and reliability in standard scenarios such as daytime scenes with favorable illumination conditions. However, in face of adverse conditions such as the nighttime, object detection loses its accuracy signific…
Department of Information Engineering and Mathematics, University of Siena（锡耶纳大学信息工程与数学系）；IsTech s.r.l.（IsTech股份有限公司(IsTech s.r.l.)）
In this paper we describe a video surveillance system able to detect traffic events in videos acquired by fixed videocameras on highways. The events of interest consist in a specific sequence of situations that occur in the video, as for instance a vehicle stopping on the emergency lane. Hence, the detection of these events requires to analyze a temporal sequence in the video stream. We compare different approaches that exploit architectures based on Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs). A first approach extracts vectors of features, mostly related to motio…
Vehicle detection is a technology which its aim is to locate and show the vehicle size in digital images. In this technology, vehicles are detected in presence of other things like trees and buildings. It has an important role in many computer vision applications such as vehicle tracking, analyzing the traffic scene and efficient traffic management. In this paper, vehicles detected based on the boosting technique by Viola Jones. Our proposed system is tested in some real scenes of surveillance videos with different light conditions. The experimental results show that the accuracy,completeness,…
In this paper we propose a traffic surveillance camera calibration method based on detection of pairs of vanishing points associated with vehicles in the traffic surveillance footage. To detect the vanishing points we propose a CNN which outputs heatmaps in which the positions of vanishing points are represented using the diamond space parametrization which enables us to detect vanishing points from the whole infinite projective space. From the detected pairs of vanishing points for multiple vehicles in a scene we establish the scene geometry by estimating the focal length of the camera and th…
University of Missouri-Columbia（密苏里大学哥伦比亚分校）；WSP USA（美国WSP公司）
Manual traffic surveillance can be a daunting task as Traffic Management Centers operate a myriad of cameras installed over a network. Injecting some level of automation could help lighten the workload of human operators performing manual surveillance and facilitate making proactive decisions which would reduce the impact of incidents and recurring congestion on roadways. This article presents a novel approach to automatically monitor real time traffic footage using deep convolutional neural networks and a stand-alone graphical user interface. The authors describe the results of research recei…
University Technology Malaysia（马来西亚工艺大学）；Iraqi University（伊拉克大学）；Baghdad University（巴格达大学）
Vehicle detection and tracking applications play an important role for civilian and military applications such as in highway traffic surveillance control, management and urban traffic planning. Vehicle detection process on road are used for vehicle tracking, counts, average speed of each individual vehicle, traffic analysis and vehicle categorizing objectives and may be implemented under different environments changes. In this review, we present a concise overview of image processing methods and analysis tools which used in building these previous mentioned applications that involved developin…
Surveillance Video-Based Traffic Accident Detection Using Transformer Architecture
Department of Civil Engineering, Indian Institute of Technology Kanpur（印度理工学院坎浦尔分校土木工程系）；School of Computing, Engineering and Mathematical Sciences, La Trobe University（拉特罗布大学计算科学与工程数学科学学院）
Department of Civil, Architectural, and Environmental Engineering, University of Texas at Austin（德克萨斯大学奥斯汀分校土木、建筑与环境工程系）；Faculty of Electrical and Computer Engineering, Babol Noshirvani University of Technology（巴布里·纳希里万理工大学电气与计算机工程学院）；Transportation Research Institute (IMOB), UHasselt-Hasselt University（乌赫斯泰特大学交通研究所）；Faculty of Mechanical Engineering, Babol Noshirvani University of Technology（巴布里·纳希里万理工大学机械工程学院）；Department of Computer Engineering, Amirkabir University of Technology（阿姆irkabir技术大学计算机工程系）
Visual Analysis of People Laboratory（视觉分析实验室）；Aalborg University（奥尔堡大学）
In integrated surveillance systems based on visual cameras, the mitigation of adverse weather conditions is an active research topic. Within this field, rain removal algorithms have been developed that artificially remove rain streaks from images or video. In order to deploy such rain removal algorithms in a surveillance setting, one must detect if rain is present in the scene. In this paper, we design a system for the detection of rainfall by the use of surveillance cameras. We reimplement the former state-of-the-art method for rain detection and compare it against a modern CNN-based method b…
Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Survey
School of Automation, Nanjing University of Science and Technology（南京理工大学自动化学院）；School of Transportation, Southeast University（东南大学交通学院）；Beijing Laboratory of General Aviation Technology, Beijing University of Civil Engineering and Architecture（北京建筑大学通用航空技术实验室）
Saab-NTU Joint Lab（萨博-南洋理工大学联合实验室）；Nanyang Technological University（南洋理工大学）；Air Traffic Management Research Institute（空中交通管理研究院）；VNUHCM-University of Science（越南胡志明市国家大学科学大学）
An airport runway and taxiway (airside) area is a highly dynamic and complex environment featuring interactions between different types of vehicles (speed and dimension), under varying visibility and traffic conditions. Airport ground movements are deemed safety-critical activities, and safe-separation procedures must be maintained by Air Traffic Controllers (ATCs). Large airports with complicated runway-taxiway systems use advanced ground surveillance systems. However, these systems have inherent limitations and a lack of real-time analytics. In this paper, we propose a novel computer-vision …
Smart automated traffic enforcement solutions have been gaining popularity in recent years. These solutions are ubiquitously used for seat-belt violation detection, red-light violation detection and speed violation detection purposes. Highly accurate license plate recognition is an indispensable part of these systems. However, general license plate recognition systems require high resolution images for high performance. In this study, we propose a novel license plate recognition method for general roadway surveillance cameras. Proposed segmentation free license plate recognition algorithm util…
Institute for Transport Studies, The University of Leeds（利兹大学交通研究所）；Qazvin University（加兹温大学）；Tehran Azad University, Science & Research Branch（德黑兰阿扎德大学科学与研究分院）
Computer Vision has played a major role in Intelligent Transportation Systems (ITS) and traffic surveillance. Along with the rapidly growing automated vehicles and crowded cities, the automated and advanced traffic management systems (ATMS) using video surveillance infrastructures have been evolved by the implementation of Deep Neural Networks. In this research, we provide a practical platform for real-time traffic monitoring, including 3D vehicle/pedestrian detection, speed detection, trajectory estimation, congestion detection, as well as monitoring the interaction of vehicles and pedestrian…
University of Cincinnati（辛辛那提大学）；Central Michigan University（中央密歇根大学）；University of Louisiana at Lafayette（路易斯安那大学拉斐特分校）
Action detection and public traffic safety are crucial aspects of a safe community and a better society. Monitoring traffic flows in a smart city using different surveillance cameras can play a significant role in recognizing accidents and alerting first responders. The utilization of action recognition (AR) in computer vision tasks has contributed towards high-precision applications in video surveillance, medical imaging, and digital signal processing. This paper presents an intensive review focusing on action recognition in accident detection and autonomous transportation systems for a smart…
Idiap Research Institute（伊迪亚普研究所）；Ecole Polytechnique Federale de Lausanne (EPFL)（洛桑联邦理工学院）；Brno University of Technology（布尔诺理工大学）；ReplayWell（ReplayWell公司）
Air traffic management and specifically air-traffic control (ATC) rely mostly on voice communications between Air Traffic Controllers (ATCos) and pilots. In most cases, these voice communications follow a well-defined grammar that could be leveraged in Automatic Speech Recognition (ASR) technologies. The callsign used to address an airplane is an essential part of all ATCo-pilot communications. We propose a two-steps approach to add contextual knowledge during semi-supervised training to reduce the ASR system error rates at recognizing the part of the utterance that contains the callsign. Init…
University of Washington（华盛顿大学）；University of Technology, Sydney（悉尼科技大学）；Birkbeck College, University of London（伦敦大学伯贝克学院）
Technological advances in sensor manufacture, communication, and computing are stimulating the development of new applications that are transforming traditional vision systems into pervasive intelligent camera networks. The analysis of visual cues in multi-camera networks enables a wide range of applications, from smart home and office automation to large area surveillance and traffic surveillance. While dense camera networks - in which most cameras have large overlapping fields of view - are well studied, we are mainly concerned with sparse camera networks. A sparse camera network undertakes …
Detecting and tracking vehicles in urban scenes is a crucial step in many traffic-related applications as it helps to improve road user safety among other benefits. Various challenges remain unresolved in multi-object tracking (MOT) including target information description, long-term occlusions and fast motion. We propose a multi-vehicle detection and tracking system following the tracking-by-detection paradigm that tackles the previously mentioned challenges. Our MOT method extends an Intersection-over-Union (IOU)-based tracker with vehicle re-identification features. This allows us to utiliz…
University of Michigan（密歇根大学）；Ford Motor Company（福特汽车公司）
We propose a novel and pragmatic framework for traffic scene perception with roadside cameras. The proposed framework covers a full-stack of roadside perception pipeline for infrastructure-assisted autonomous driving, including object detection, object localization, object tracking, and multi-camera information fusion. Unlike previous vision-based perception frameworks rely upon depth offset or 3D annotation at training, we adopt a modular decoupling design and introduce a landmark-based 3D localization method, where the detection and localization can be well decoupled so that the model can be…
