With the railway transportation Industry moving actively towards automation, accurate location and inventory of wayside track assets like traffic signals, crossings, switches, mileposts, etc. is of extreme importance. With the new Positive Train Control (PTC) regulation coming into effect, many railway safety rules will be tied directly to location of assets like mileposts and signals. Newer speed regulations will be enforced based on location of the Train with respect to a wayside asset. Hence it is essential for the railroads to have an accurate database of the types and locations of these assets. This paper talks about a real-world use-case of detecting railway signals from a camera mounted on a moving locomotive and tracking their locations. The camera is engineered to withstand the environment factors on a moving train and provide a consistent steady image at around 30 frames per second. Using advanced image analysis and deep learning techniques, signals are detected in these camera images and a database of their locations is created. Railway signals differ a lot from road signals in terms of shapes and rules for placement with respect to track. Due to space constraint and traffic densities in urban areas signals are not placed on the same side of the track and multiple lines can run in parallel. Hence there is need to associate signal detected with the track on which the train runs. We present a method to associate the signals to the specific track they belong to using a video feed from the front facing camera mounted on the lead locomotive. A pipeline of track detection, region of interest selection, signal detection has been implemented which gives an overall accuracy of 94.7% on a route covering 150km with 247 signals.
Computer vision based methods have been explored in the past for detection of railway track defects, but full automation has always been a challenge because both traditional image processing methods and deep learning classifiers trained from scratch fail to generalize that well to infinite novel scenarios seen in the real world, given limited amount of labeled data. Advancements have been made recently to make machine learning models utilize knowledge from a different but related domain. In this paper, we show that even though similar domain data is not available, transfer learning provides th…
Deep Learning methods have been extensively used to analyze video data to extract valuable information by classifying image frames and detecting objects. We describe a unique approach for using video feed from a moving Locomotive to continuously monitor the Railway Track and detect significant assets like Switches on the Track. The technique used here is called Siamese Networks, which uses 2 identical networks to learn the similarity between of 2 images. Here we will use a Siamese network to continuously compare Track images and detect any significant difference in the Track. Switch will be on…
Railroad tracks need to be periodically inspected and monitored to ensure safe transportation. Automated track inspection using computer vision and pattern recognition methods have recently shown the potential to improve safety by allowing for more frequent inspections while reducing human errors. Achieving full automation is still very challenging due to the number of different possible failure modes as well as the broad range of image variations that can potentially trigger false alarms. Also, the number of defective components is very small, so not many training examples are available for t…
Regular maintenance of all the assets is pivotal for proper functioning of railway. Manual maintenance can be very cumbersome and leave room for errors. Track anomalies like vegetation overgrowth, sun kinks affect the track construct and result in unequal load transfer, imbalanced lateral forces on tracks which causes further deterioration of tracks and can ultimately result in derailment of locomotive. Hence there is a need to continuously monitor rail track health. Track anomalies are rare with the skew as high as one anomaly in millions of good images. We propose a method to build training …
Periodic inspections are necessary to keep railroad tracks in state of good repair and prevent train accidents. Automatic track inspection using machine vision technology has become a very effective inspection tool. Because of its non-contact nature, this technology can be deployed on virtually any railway vehicle to continuously survey the tracks and send exception reports to track maintenance personnel. However, as appearance and imaging conditions vary, false alarm rates can dramatically change, making it difficult to select a good operating point. In this paper, we use extreme value theory…
Phys-3D: Physics-Constrained Real-Time Crowd Tracking and Counting on Railway Platforms
Visual Computing Group, Humboldt University Berlin（柏林洪堡大学视觉计算组）；Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI（弗劳恩霍夫电信研究所，海因里希·赫兹研究所，HHI）
Institute of Land and Sea Transport Systems, Technical University of Berlin（柏林工业大学陆海交通系统研究所）；Faculty of Transport Engineering, Vilnius Gediminas Technical University（维尔纽斯格迪米纳斯技术大学交通工程学院）；Department of Buildings and Environmental Engineering, Sapienza University of Rome（罗马大学建筑与环境工程系）
In the application of computer-vision based displacement measurement, an optical target is usually required to prove the reference. In the case that the optical target cannot be attached to the measuring objective, edge detection, feature matching and template matching are the most common approaches in target-less photogrammetry. However, their performance significantly relies on parameter settings. This becomes problematic in dynamic scenes where complicated background texture exists and varies over time. To tackle this issue, we propose virtual point tracking for real-time target-less dynami…
Improved YOLOv5 Based on Attention Mechanism and FasterNet for Foreign Object Detection on Railway and Airway tracks
Stevens Institute of Technology（史蒂文斯理工学院）；Northern Arizona University（北亚利桑那大学）；University of Electronic Science and Technology of China（电子科技大学）；Shenzhen SmartChip Microelectronics Technology Co., Ltd.（深圳市智芯微电子技术有限公司）
Virginia Polytechnic Institute and State University（弗吉尼亚理工学院暨州立大学）
Automated inspection and detection of foreign objects on railways is important for rail transportation safety as it helps prevent potential accidents and trains derailment. Most existing vision-based approaches focus on the detection of frontal intrusion objects with prior labels, such as categories and locations of the objects. In reality, foreign objects with unknown categories can appear anytime on railway tracks. In this paper, we develop a semi-supervised convolutional autoencoder based framework that only requires railway track images without prior knowledge on the foreign objects in the…
Scuola Superiore Sant’Anna（圣安娜高等学校）；Hitachi Rail STS（日立铁路STS公司）
The railway industry is searching for new ways to automate a number of complex train functions, such as object detection, track discrimination, and accurate train positioning, which require the artificial perception of the railway environment through different types of sensors, including cameras, LiDARs, wheel encoders, and inertial measurement units. A promising approach for processing such sensory data is the use of deep learning models, which proved to achieve excellent performance in other application domains, as robotics and self-driving cars. However, testing new algorithms and solutions…
This document describes the theory used for the calculation of track geometric irregularities on a Track Geometry Measuring System (TGMS) to be installed in railway vehicles. The TGMS includes a computer for data acquisition and process, a set of sensors including an inertial measuring unit (IMU, 3D gyroscope and 3D accelerometer), two video cameras and an encoder. The main features of the proposed system are: 1. It is capable to measure track alignment, vertical profile, cross-level, gauge, twist and rail-head profile using non-contact technology. 2. It can be installed in line railway vehicl…
Institut de Recherche Technologique SystemX（法国系统X技术研究所）；Société Nationale des Chemins de Fer（法国国家铁路公司）；Alstom（阿尔斯通公司(Alstom)）
In the realm of autonomous transportation, there have been many initiatives for open-sourcing self-driving cars datasets, but much less for alternative methods of transportation such as trains. In this paper, we aim to bridge the gap by introducing FRSign, a large-scale and accurate dataset for vision-based railway traffic light detection and recognition. Our recordings were made on selected running trains in France and benefited from carefully hand-labeled annotations. An illustrative dataset which corresponds to ten percent of the acquired data to date is published in open source with the pa…
Train Pilot is a very tedious and stressful job. Pilots must be vigilant at all times and its easy for them to lose track of time of shift. In countries like USA the pilots are mandated by law to adhere to 8 hour shifts. If they exceed 8 hours of shift the railroads may be penalized for over-tiring their drivers. The problem happens when the 8 hour shift may end in middle of a journey. In such case, the new drivers must be moved to the location locomotive is operating for shift change. Hence accurate monitoring of drivers during their shift and making sure the shifts are scheduled correctly is…
Development of computing power and cheap video cameras enabled today's traffic management systems to include more cameras and computer vision applications for transportation system monitoring and control. Combined with image processing algorithms cameras are used as sensors to measure road traffic parameters like flow volume, origin-destination matrices, classify vehicles, etc. In this paper we propose a system for measurement of road traffic parameters (basic motion model parameters and macro-scopic traffic parameters). The system is based on Local Binary Pattern (LBP) image features classifi…
Multi-target multi-camera tracking (MTMCT), i.e., tracking multiple targets across multiple cameras, is a crucial technique for smart city applications. In this paper, we propose an effective and reliable MTMCT framework for vehicles, which consists of a traffic-aware single camera tracking (TSCT) algorithm, a trajectory-based camera link model (CLM) for vehicle re-identification (ReID), and a hierarchical clustering algorithm to obtain the cross camera vehicle trajectories. First, the TSCT, which jointly considers vehicle appearance, geometric features, and some common traffic scenarios, is p…
Urban Traffic Surveillance (UTS) is a surveillance system based on a monocular and calibrated video camera that detects vehicles in an urban traffic scenario with dense traffic on multiple lanes and vehicles performing sharp turning maneuvers. UTS then tracks the vehicles using a 3D bounding box representation and a physically reasonable 3D motion model relying on an unscented Kalman filter based approach. Since UTS recovers positions, shape and motion information in a three-dimensional world coordinate system, it can be employed to recognize diverse traffic violations or to supply intelligent…
This paper presents the learned techniques during the Video Analysis Module of the Master in Computer Vision from the Universitat Autònoma de Barcelona, used to solve the third track of the AI-City Challenge. This challenge aims to track vehicles across multiple cameras placed in multiple intersections spread out over a city. The methodology followed focuses first in solving multi-tracking in a single camera and then extending it to multiple cameras. The qualitative results of the implemented techniques are presented using standard metrics for video analysis such as mAP for object detection an…
TUMTraf EMOT: Event-Based Multi-Object Tracking Dataset and Baseline for Traffic Scenarios
School of Automation, Southeast University（东南大学自动化学院）；Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich（慕尼黑技术大学机器人与人工智能与实时系统教授职位）；Department of Computer Science and Technology, Tongji University（同济大学计算机科学与技术系）
Railway LiDAR semantic segmentation based on intelligent semi-automated data annotation
Fraunhofer Institute for Open Communication Technologies (FOKUS)（弗劳恩霍夫开放通信技术研究所）；Daimler Center for Automotive Information Technology Innovations (DCAITI)（戴姆勒汽车信息技术创新中心）
Indian Institute of Technology Bhubaneswar（印度理工学院布巴内斯瓦尔分校）；Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）
Accurate prediction of traffic signal duration for roadway junction is a challenging problem due to the dynamic nature of traffic flows. Though supervised learning can be used, parameters may vary across roadway junctions. In this paper, we present a computer vision guided expert system that can learn the departure rate of a given traffic junction modeled using traditional queuing theory. First, we temporally group the optical flow of the moving vehicles using Dirichlet Process Mixture Model (DPMM). These groups are referred to as tracklets or temporal clusters. Tracklet features are then used…
Tracking without Seeing: Geospatial Inference using Encrypted Traffic from Distributed Nodes
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）；DEVCOM Army Research Laboratory（DEVCOM陆军研究实验室）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
