Partially Detected Intelligent Traffic Signal Control (PD-ITSC) systems that can optimize traffic signals based on limited detected information could be a cost-efficient solution for mitigating traffic congestion in the future. In this paper, we focus on a particular problem in PD-ITSC - adaptation to changing environments. To this end, we investigate different reinforcement learning algorithms, including Q-learning, Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Actor-Critic with Kronecker-Factored Trust Region (ACKTR). Our findings suggest that RL algorithms can find optimal strategies under partial vehicle detection; however, policy-based algorithms can adapt to changing environments more efficiently than value-based algorithms. We use these findings to draw conclusions about the value of different models for PD-ITSC systems.
Pakistan Institute of Engineering and Applied Sciences（巴基斯坦工程与应用科学学院）；Carleton University（卡尔顿大学）
Deep Reinforcement Learning (DRL) uses diverse, unstructured data and makes RL capable of learning complex policies in high dimensional environments. Intelligent Transportation System (ITS) based on Autonomous Vehicles (AVs) offers an excellent playground for policy-based DRL. Deep learning architectures solve computational challenges of traditional algorithms while helping in real-world adoption and deployment of AVs. One of the main challenges in AVs implementation is that it can worsen traffic congestion on roads if not reliably and efficiently managed. Considering each vehicle's holistic e…
The University of Iowa（爱荷华大学）；National Advanced Driving Simulator, The University of Iowa（爱荷华大学国家高级驾驶模拟器）；Iowa Technology Institute, The University of Iowa（爱荷华大学爱荷华技术研究所）
Intelligent traffic lights in smart cities can optimally reduce traffic congestion. In this study, we employ reinforcement learning to train the control agent of a traffic light on a simulator of urban mobility. As a difference from existing works, a policy-based deep reinforcement learning method, Proximal Policy Optimization (PPO), is utilized other than value-based methods such as Deep Q Network (DQN) and Double DQN (DDQN). At first, the obtained optimal policy from PPO is compared to those from DQN and DDQN. It is found that the policy from PPO performs better than the others. Next, instea…
The recent advancements in cloud services, Internet of Things (IoT) and Cellular networks have made cloud computing an attractive option for intelligent traffic signal control (ITSC). Such a method significantly reduces the cost of cables, installation, number of devices used, and maintenance. ITSC systems based on cloud computing lower the cost of the ITSC systems and make it possible to scale the system by utilizing the existing powerful cloud platforms. While such systems have significant potential, one of the critical problems that should be addressed is the network delay. It is well known…
Intelligent Traffic Light Control System (ITLCS) is a typical Multi-Agent System (MAS), which comprises multiple roads and traffic lights.Constructing a model of MAS for ITLCS is the basis to alleviate traffic congestion. Existing approaches of MAS are largely based on Multi-Agent Deep Reinforcement Learning (MADRL). Although the Deep Neural Network (DNN) of MABRL is effective, the training time is long, and the parameters are difficult to trace. Recently, Broad Learning Systems (BLS) provided a selective way for learning in the deep neural networks by a flat network. Moreover, Broad Reinforce…
Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Electrical Engineering, Columbia University（哥伦比亚大学电气工程系）；Nokia Bell Labs（诺基亚贝尔实验室）
SIGMA: Symmetry-aware, Intelligent, Geometric, Multi-faced Adaptive Traffic Control for Reliable Intelligent Transportation Systems
Indian Statistical Institute（印度统计研究所）；Sorbonne University Abu Dhabi（阿布扎比索邦大学）；Sorbonne Center for Artificial Intelligence（索邦人工智能中心）；SQC & OR（统计质量控制与运筹学研究室）
Siemens Technology and Services Pvt. Ltd.（西门子技术服务私人有限公司）；International Institute of Information Technology Hyderabad（海得拉巴国际信息技术学院）；Indian Institute of Science Bengaluru（班加罗尔印度科学研究所）
We use Asynchronous Advantage Actor Critic (A3C) for implementing an AI agent in the controllers that optimize flow of traffic across a single intersection and then extend it to multiple intersections by considering a multi-agent setting. We explore three different methodologies to address the multi-agent problem - (1) use of asynchronous property of A3C to control multiple intersections using a single agent (2) utilise self/competitive play among independent agents across multiple intersections and (3) ingest a global reward function among agents to introduce cooperative behavior between inte…
In this paper, methods have been explored to effectively optimise traffic signal control to minimise waiting times and queue lengths, thereby increasing traffic flow. The traffic intersection was first defined as a Markov Decision Process, and a state representation, actions and rewards were chosen. Simulation of Urban MObility (SUMO) was used to simulate an intersection and then compare a Round Robin Scheduler, a Feedback Control mechanism and two Reinforcement Learning techniques - Deep Q Network (DQN) and Advantage Actor-Critic (A2C), as the policy for the traffic signal in the simulation u…
Intelligent traffic signal controllers, applying DQN algorithms to traffic light policy optimization, efficiently reduce traffic congestion by adjusting traffic signals to real-time traffic. Most propositions in the literature however consider that all vehicles at the intersection are detected, an unrealistic scenario. Recently, new wireless communication technologies have enabled cost-efficient detection of connected vehicles by infrastructures. With only a small fraction of the total fleet currently equipped, methods able to perform under low detection rates are desirable. In this paper, we …
The goal of Next-Generation Networks is to improve upon the current networking paradigm, especially in providing higher data rates, near-real-time latencies, and near-perfect quality of service. However, existing radio access network (RAN) architectures lack sufficient flexibility and intelligence to meet those demands. Open RAN (O-RAN) is a promising paradigm for building a virtualized and intelligent RAN architecture. This paper presents a Machine Learning (ML)-based Traffic Steering (TS) scheme to predict network congestion and then proactively steer O-RAN traffic to avoid it and reduce the…
Traffic signal control is an important and challenging real-world problem, which aims to minimize the travel time of vehicles by coordinating their movements at the road intersections. Current traffic signal control systems in use still rely heavily on oversimplified information and rule-based methods, although we now have richer data, more computing power and advanced methods to drive the development of intelligent transportation. With the growing interest in intelligent transportation using machine learning methods like reinforcement learning, this survey covers the widely acknowledged trans…
Department of Computer Science, University of Memphis（孟菲斯大学计算机科学系）；University of Memphis（孟菲斯大学）
We present a novel adaptive cruise control (ACC) system namely SAINT-ACC: {S}afety-{A}ware {Int}elligent {ACC} system (SAINT-ACC) that is designed to achieve simultaneous optimization of traffic efficiency, driving safety, and driving comfort through dynamic adaptation of the inter-vehicle gap based on deep reinforcement learning (RL). A novel dual RL agent-based approach is developed to seek and adapt the optimal balance between traffic efficiency and driving safety/comfort by effectively controlling the driving safety model parameters and inter-vehicle gap based on macroscopic and microscopi…
Columbia University（哥伦比亚大学）；Imperial College London（帝国理工学院）；Nokia-Bell Laboratories（诺基亚贝尔实验室）
Intelligent Transportation Systems (ITSs) are envisioned to play a critical role in improving traffic flow and reducing congestion, which is a pervasive issue impacting urban areas around the globe. Rapidly advancing vehicular communication and edge cloud computation technologies provide key enablers for smart traffic management. However, operating viable real-time actuation mechanisms on a practically relevant scale involves formidable challenges, e.g., policy iteration and conventional Reinforcement Learning (RL) techniques suffer from poor scalability due to state space explosion. Motivated…
Traffic control using intelligent timing of traffic lights with reinforcement learning technique and real-time processing of surveillance camera images
Today's intelligent traffic light control system is based on the current road traffic conditions for traffic regulation. However, these approaches cannot exploit the future traffic information in advance. In this paper, we propose GPlight, a deep reinforcement learning (DRL) algorithm integrated with graph neural network (GNN) , to relieve the traffic congestion for multi-intersection intelligent traffic control system. In GPlight, the graph neural network (GNN) is first used to predict the future short-term traffic flow at the intersections. Then, the results of traffic flow prediction are us…
Radio Resource Management and Path Planning in Intelligent Transportation Systems via Reinforcement Learning for Environmental Sustainability
Tarbiat Modares University（塔比亚特·莫达雷斯大学）；Doha University for Science and Technology（多哈科技大学）；TU Braunschweig（布伦瑞克工业大学）；Institute of Communications Technology（通信技术研究所）
Intelligent Transportation Systems (ITS) have attracted the attention of researchers and the general public alike as a means to alleviate traffic congestion. Recently, the maturity of wireless technology has enabled a cost-efficient way to achieve ITS by detecting vehicles using Vehicle to Infrastructure (V2I) communications. Traditional ITS algorithms, in most cases, assume that every vehicle is observed, such as by a camera or a loop detector, but a V2I implementation would detect only those vehicles with wireless communications capability. We examine a family of transportation systems, whic…
This paper studies the traffic monitoring problem in a road network using a team of aerial robots. The problem is challenging due to two main reasons. First, the traffic events are stochastic, both temporally and spatially. Second, the problem has a non-homogeneous structure as the traffic events arrive at different locations of the road network at different rates. Accordingly, some locations require more visits by the robots compared to other locations. To address these issues, we define an uncertainty metric for each location of the road network and formulate a path planning problem for the …
GARLIC: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching
City University of Hong Kong（香港城市大学）；Jilin University（吉林大学）；Zhejiang University of Technology（浙江工业大学）；Southern University of Science and Technology（南方科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Alibaba Group（阿里巴巴集团）
Universitat Pompeu Fabra（庞培法布拉大学）；Inria（法国国家信息与自动化研究所）；Université de Bordeaux（波尔多大学）；ENSTA ParisTech（巴黎高等先进技术学院）；Institució Catalana de Recerca i Estudis Avançats（加泰罗尼亚研究与高等研究院）；Santa Fe Institute（圣塔菲研究所）
Humans have been able to tackle biosphere complexities by acting as ecosystem engineers, profoundly changing the flows of matter, energy and information. This includes major innovations that allowed to reduce and control the impact of extreme events. Modelling the evolution of such adaptive dynamics can be challenging given the potentially large number of individual and environmental variables involved. This paper shows how to address this problem by using fire as the source of external, bursting and wide fluctuations. Fire propagates on a spatial landscape where a group of agents harvest and …
Georgia Institute of Technology（佐治亚理工学院）；CableLabs, Inc.（CableLabs公司）
A novel intelligent bandwidth allocation scheme in NG-EPON using reinforcement learning is proposed and demonstrated for latency management. We verify the capability of the proposed scheme under both fixed and dynamic traffic loads scenarios to achieve <1ms average latency. The RL agent demonstrates an efficient intelligent mechanism to manage the latency, which provides a promising IBA solution for the next-generation access network.
Most traffic flow control algorithms address switching cycle adaptation of traffic signals and lights. This work addresses traffic flow optimisation by self-organising micro-level control combining Reinforcement Learning and rule-based agents for action selection performing long-range navigation in urban environments. I.e., vehicles represented by agents adapt their decision making for re-routing based on local environmental sensors. Agent-based modelling and simulation is used to study emergence effects on urban city traffic flows. An unified agent programming model enables simulation and dis…
Network-Aided Intelligent Traffic Steering in 6G O-RAN: A Multi-Layer Optimization Framework
VinUniversity（VinUniversity（文大学））；University of Luxembourg（卢森堡大学）；University of Oulu（奥卢大学）；PHENIKAA University（PHENIKAA大学（菲尼卡大学））；Purdue University（普渡大学）；University of Technology Sydney（悉尼科技大学）
Latest technological improvements increased the quality of transportation. New data-driven approaches bring out a new research direction for all control-based systems, e.g., in transportation, robotics, IoT and power systems. Combining data-driven applications with transportation systems plays a key role in recent transportation applications. In this paper, the latest deep reinforcement learning (RL) based traffic control applications are surveyed. Specifically, traffic signal control (TSC) applications based on (deep) RL, which have been studied extensively in the literature, are discussed in…
Connected Autonomous Vehicles will make autonomous intersection management a reality replacing traditional traffic signal control. Autonomous intersection management requires time and speed adjustment of vehicles arriving at an intersection for collision-free passing through the intersection. Due to its computational complexity, this problem has been studied only when vehicle arrival times towards the vicinity of the intersection are known beforehand, which limits the applicability of these solutions for real-time deployment. To solve the real-time autonomous traffic intersection management pr…
University of Engineering and Technology Peshawar（白沙瓦工程技术大学）；University of Swabi（斯瓦比大学）；COMSATS University Islamabad（伊斯兰堡COMSATS大学）；Jessore University of Science and Technology（杰索尔科技大学）
Vehicle speed monitoring and management of highways is the critical problem of the road in this modern age of growing technology and population. A poor management results in frequent traffic jam, traffic rules violation and fatal road accidents. Using traditional techniques of RADAR, LIDAR and LASAR to address this problem is time-consuming, expensive and tedious. This paper presents an efficient framework to produce a simple, cost efficient and intelligent system for vehicle speed monitoring. The proposed method uses an HD (High Definition) camera mounted on the road side either on a pole or …
A traffic monitoring system is an integral part of Intelligent Transportation Systems (ITS). It is one of the critical transportation infrastructures that transportation agencies invest a huge amount of money to collect and analyze the traffic data to better utilize the roadway systems, improve the safety of transportation, and establish future transportation plans. With recent advances in MEMS, machine learning, and wireless communication technologies, numerous innovative traffic monitoring systems have been developed. In this article, we present a review of state-of-the-art traffic monitorin…
This paper introduces Sentinel, an onboard system for intelligent vehicles that guides their lane changing behavior during a freeway incident with the goal of reducing traffic congestion, capacity drop, and delay. When an incident blocking the lanes ahead is detected, Sentinel calculates the probability of leaving the blocked lane(s) before reaching the incident point at each time step. It advises the vehicle to leave the blocked lane(s) when that probability drops below a certain threshold, as the vehicle nears the congestion boundary. By doing this, Sentinel reduces the number of late-stage …
An Adaptive Cruise Control (ACC) system allows vehicles to maintain a desired headway distance to a preceding vehicle automatically. It is increasingly adopted by commercial vehicles. Recent research demonstrates that the effective use of ACC can improve the traffic flow through the adaptation of the headway distance in response to the current traffic conditions. In this paper, we demonstrate that a state-of-the-art intelligent ACC system performs poorly on highways with ramps due to the limitation of the model-based approaches that do not take into account appropriately the traffic dynamics o…
Aalborg University（奥尔堡大学）；Software Radio Systems（软件无线电系统公司）
Environmental scene reconstruction is of great interest for autonomous robotic applications, since an accurate representation of the environment is necessary to ensure safe interaction with robots. Equally important, it is also vital to ensure reliable communication between the robot and its controller. Large Intelligent Surface (LIS) is a technology that has been extensively studied due to its communication capabilities. Moreover, due to the number of antenna elements, these surfaces arise as a powerful solution to radio sensing. This paper presents a novel method to translate radio environme…
