The recent advancements in cloud services, Internet of Things (IoT) and Cellular networks have made cloud computing an attractive option for intelligent traffic signal control (ITSC). Such a method significantly reduces the cost of cables, installation, number of devices used, and maintenance. ITSC systems based on cloud computing lower the cost of the ITSC systems and make it possible to scale the system by utilizing the existing powerful cloud platforms.
While such systems have significant potential, one of the critical problems that should be addressed is the network delay. It is well known that network delay in message propagation is hard to prevent, which could potentially degrade the performance of the system or even create safety issues for vehicles at intersections.
In this paper, we introduce a new traffic signal control algorithm based on reinforcement learning, which performs well even under severe network delay. The framework introduced in this paper can be helpful for all agent-based systems using remote computing resources where network delay could be a critical concern. Extensive simulation results obtained for different scenarios show the viability of the designed algorithm to cope with network delay.
Carnegie Mellon University（卡内基梅隆大学）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；King Abdulaziz City for Science and Technology（阿卜杜勒阿齐兹国王科技城）
Partially Detected Intelligent Traffic Signal Control (PD-ITSC) systems that can optimize traffic signals based on limited detected information could be a cost-efficient solution for mitigating traffic congestion in the future. In this paper, we focus on a particular problem in PD-ITSC - adaptation to changing environments. To this end, we investigate different reinforcement learning algorithms, including Q-learning, Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Actor-Critic with Kronecker-Factored Trust Region (ACKTR). Our findings suggest that RL algorithms can find o…
Pakistan Institute of Engineering and Applied Sciences（巴基斯坦工程与应用科学学院）；Carleton University（卡尔顿大学）
Deep Reinforcement Learning (DRL) uses diverse, unstructured data and makes RL capable of learning complex policies in high dimensional environments. Intelligent Transportation System (ITS) based on Autonomous Vehicles (AVs) offers an excellent playground for policy-based DRL. Deep learning architectures solve computational challenges of traditional algorithms while helping in real-world adoption and deployment of AVs. One of the main challenges in AVs implementation is that it can worsen traffic congestion on roads if not reliably and efficiently managed. Considering each vehicle's holistic e…
Traffic control using intelligent timing of traffic lights with reinforcement learning technique and real-time processing of surveillance camera images
Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Electrical Engineering, Columbia University（哥伦比亚大学电气工程系）；Nokia Bell Labs（诺基亚贝尔实验室）
Siemens Technology and Services Pvt. Ltd.（西门子技术服务私人有限公司）；International Institute of Information Technology Hyderabad（海得拉巴国际信息技术学院）；Indian Institute of Science Bengaluru（班加罗尔印度科学研究所）
We use Asynchronous Advantage Actor Critic (A3C) for implementing an AI agent in the controllers that optimize flow of traffic across a single intersection and then extend it to multiple intersections by considering a multi-agent setting. We explore three different methodologies to address the multi-agent problem - (1) use of asynchronous property of A3C to control multiple intersections using a single agent (2) utilise self/competitive play among independent agents across multiple intersections and (3) ingest a global reward function among agents to introduce cooperative behavior between inte…
Intelligent Transportation Systems (ITS) have attracted the attention of researchers and the general public alike as a means to alleviate traffic congestion. Recently, the maturity of wireless technology has enabled a cost-efficient way to achieve ITS by detecting vehicles using Vehicle to Infrastructure (V2I) communications. Traditional ITS algorithms, in most cases, assume that every vehicle is observed, such as by a camera or a loop detector, but a V2I implementation would detect only those vehicles with wireless communications capability. We examine a family of transportation systems, whic…
Traffic signal control is an important and challenging real-world problem, which aims to minimize the travel time of vehicles by coordinating their movements at the road intersections. Current traffic signal control systems in use still rely heavily on oversimplified information and rule-based methods, although we now have richer data, more computing power and advanced methods to drive the development of intelligent transportation. With the growing interest in intelligent transportation using machine learning methods like reinforcement learning, this survey covers the widely acknowledged trans…
This paper studies the traffic monitoring problem in a road network using a team of aerial robots. The problem is challenging due to two main reasons. First, the traffic events are stochastic, both temporally and spatially. Second, the problem has a non-homogeneous structure as the traffic events arrive at different locations of the road network at different rates. Accordingly, some locations require more visits by the robots compared to other locations. To address these issues, we define an uncertainty metric for each location of the road network and formulate a path planning problem for the …
GARLIC: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching
City University of Hong Kong（香港城市大学）；Jilin University（吉林大学）；Zhejiang University of Technology（浙江工业大学）；Southern University of Science and Technology（南方科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Alibaba Group（阿里巴巴集团）
A traffic monitoring system is an integral part of Intelligent Transportation Systems (ITS). It is one of the critical transportation infrastructures that transportation agencies invest a huge amount of money to collect and analyze the traffic data to better utilize the roadway systems, improve the safety of transportation, and establish future transportation plans. With recent advances in MEMS, machine learning, and wireless communication technologies, numerous innovative traffic monitoring systems have been developed. In this article, we present a review of state-of-the-art traffic monitorin…
University of Engineering and Technology Peshawar（白沙瓦工程技术大学）；University of Swabi（斯瓦比大学）；COMSATS University Islamabad（伊斯兰堡COMSATS大学）；Jessore University of Science and Technology（杰索尔科技大学）
Vehicle speed monitoring and management of highways is the critical problem of the road in this modern age of growing technology and population. A poor management results in frequent traffic jam, traffic rules violation and fatal road accidents. Using traditional techniques of RADAR, LIDAR and LASAR to address this problem is time-consuming, expensive and tedious. This paper presents an efficient framework to produce a simple, cost efficient and intelligent system for vehicle speed monitoring. The proposed method uses an HD (High Definition) camera mounted on the road side either on a pole or …
This paper introduces Sentinel, an onboard system for intelligent vehicles that guides their lane changing behavior during a freeway incident with the goal of reducing traffic congestion, capacity drop, and delay. When an incident blocking the lanes ahead is detected, Sentinel calculates the probability of leaving the blocked lane(s) before reaching the incident point at each time step. It advises the vehicle to leave the blocked lane(s) when that probability drops below a certain threshold, as the vehicle nears the congestion boundary. By doing this, Sentinel reduces the number of late-stage …
An Adaptive Cruise Control (ACC) system allows vehicles to maintain a desired headway distance to a preceding vehicle automatically. It is increasingly adopted by commercial vehicles. Recent research demonstrates that the effective use of ACC can improve the traffic flow through the adaptation of the headway distance in response to the current traffic conditions. In this paper, we demonstrate that a state-of-the-art intelligent ACC system performs poorly on highways with ramps due to the limitation of the model-based approaches that do not take into account appropriately the traffic dynamics o…
Latest technological improvements increased the quality of transportation. New data-driven approaches bring out a new research direction for all control-based systems, e.g., in transportation, robotics, IoT and power systems. Combining data-driven applications with transportation systems plays a key role in recent transportation applications. In this paper, the latest deep reinforcement learning (RL) based traffic control applications are surveyed. Specifically, traffic signal control (TSC) applications based on (deep) RL, which have been studied extensively in the literature, are discussed in…
