In this paper, methods have been explored to effectively optimise traffic signal control to minimise waiting times and queue lengths, thereby increasing traffic flow. The traffic intersection was first defined as a Markov Decision Process, and a state representation, actions and rewards were chosen. Simulation of Urban MObility (SUMO) was used to simulate an intersection and then compare a Round Robin Scheduler, a Feedback Control mechanism and two Reinforcement Learning techniques - Deep Q Network (DQN) and Advantage Actor-Critic (A2C), as the policy for the traffic signal in the simulation under different scenarios. Finally, the methods were tested on a simulation of a real-world intersection in Bengaluru, India.
Intelligent traffic signal controllers, applying DQN algorithms to traffic light policy optimization, efficiently reduce traffic congestion by adjusting traffic signals to real-time traffic. Most propositions in the literature however consider that all vehicles at the intersection are detected, an unrealistic scenario. Recently, new wireless communication technologies have enabled cost-efficient detection of connected vehicles by infrastructures. With only a small fraction of the total fleet currently equipped, methods able to perform under low detection rates are desirable. In this paper, we …
The University of Iowa（爱荷华大学）；National Advanced Driving Simulator, The University of Iowa（爱荷华大学国家高级驾驶模拟器）；Iowa Technology Institute, The University of Iowa（爱荷华大学爱荷华技术研究所）
Intelligent traffic lights in smart cities can optimally reduce traffic congestion. In this study, we employ reinforcement learning to train the control agent of a traffic light on a simulator of urban mobility. As a difference from existing works, a policy-based deep reinforcement learning method, Proximal Policy Optimization (PPO), is utilized other than value-based methods such as Deep Q Network (DQN) and Double DQN (DDQN). At first, the obtained optimal policy from PPO is compared to those from DQN and DDQN. It is found that the policy from PPO performs better than the others. Next, instea…
Network-Aided Intelligent Traffic Steering in 6G O-RAN: A Multi-Layer Optimization Framework
VinUniversity（VinUniversity（文大学））；University of Luxembourg（卢森堡大学）；University of Oulu（奥卢大学）；PHENIKAA University（PHENIKAA大学（菲尼卡大学））；Purdue University（普渡大学）；University of Technology Sydney（悉尼科技大学）
SIGMA: Symmetry-aware, Intelligent, Geometric, Multi-faced Adaptive Traffic Control for Reliable Intelligent Transportation Systems
Indian Statistical Institute（印度统计研究所）；Sorbonne University Abu Dhabi（阿布扎比索邦大学）；Sorbonne Center for Artificial Intelligence（索邦人工智能中心）；SQC & OR（统计质量控制与运筹学研究室）
Carnegie Mellon University（卡内基梅隆大学）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；King Abdulaziz City for Science and Technology（阿卜杜勒阿齐兹国王科技城）
Partially Detected Intelligent Traffic Signal Control (PD-ITSC) systems that can optimize traffic signals based on limited detected information could be a cost-efficient solution for mitigating traffic congestion in the future. In this paper, we focus on a particular problem in PD-ITSC - adaptation to changing environments. To this end, we investigate different reinforcement learning algorithms, including Q-learning, Proximal Policy Optimization (PPO), Advantage Actor-Critic (A2C), and Actor-Critic with Kronecker-Factored Trust Region (ACKTR). Our findings suggest that RL algorithms can find o…
Pakistan Institute of Engineering and Applied Sciences（巴基斯坦工程与应用科学学院）；Carleton University（卡尔顿大学）
Deep Reinforcement Learning (DRL) uses diverse, unstructured data and makes RL capable of learning complex policies in high dimensional environments. Intelligent Transportation System (ITS) based on Autonomous Vehicles (AVs) offers an excellent playground for policy-based DRL. Deep learning architectures solve computational challenges of traditional algorithms while helping in real-world adoption and deployment of AVs. One of the main challenges in AVs implementation is that it can worsen traffic congestion on roads if not reliably and efficiently managed. Considering each vehicle's holistic e…
Intelligent Traffic Light Control System (ITLCS) is a typical Multi-Agent System (MAS), which comprises multiple roads and traffic lights.Constructing a model of MAS for ITLCS is the basis to alleviate traffic congestion. Existing approaches of MAS are largely based on Multi-Agent Deep Reinforcement Learning (MADRL). Although the Deep Neural Network (DNN) of MABRL is effective, the training time is long, and the parameters are difficult to trace. Recently, Broad Learning Systems (BLS) provided a selective way for learning in the deep neural networks by a flat network. Moreover, Broad Reinforce…
The goal of Next-Generation Networks is to improve upon the current networking paradigm, especially in providing higher data rates, near-real-time latencies, and near-perfect quality of service. However, existing radio access network (RAN) architectures lack sufficient flexibility and intelligence to meet those demands. Open RAN (O-RAN) is a promising paradigm for building a virtualized and intelligent RAN architecture. This paper presents a Machine Learning (ML)-based Traffic Steering (TS) scheme to predict network congestion and then proactively steer O-RAN traffic to avoid it and reduce the…
Department of Computer Science, University of Memphis（孟菲斯大学计算机科学系）；University of Memphis（孟菲斯大学）
We present a novel adaptive cruise control (ACC) system namely SAINT-ACC: {S}afety-{A}ware {Int}elligent {ACC} system (SAINT-ACC) that is designed to achieve simultaneous optimization of traffic efficiency, driving safety, and driving comfort through dynamic adaptation of the inter-vehicle gap based on deep reinforcement learning (RL). A novel dual RL agent-based approach is developed to seek and adapt the optimal balance between traffic efficiency and driving safety/comfort by effectively controlling the driving safety model parameters and inter-vehicle gap based on macroscopic and microscopi…
Columbia University（哥伦比亚大学）；Imperial College London（帝国理工学院）；Nokia-Bell Laboratories（诺基亚贝尔实验室）
Intelligent Transportation Systems (ITSs) are envisioned to play a critical role in improving traffic flow and reducing congestion, which is a pervasive issue impacting urban areas around the globe. Rapidly advancing vehicular communication and edge cloud computation technologies provide key enablers for smart traffic management. However, operating viable real-time actuation mechanisms on a practically relevant scale involves formidable challenges, e.g., policy iteration and conventional Reinforcement Learning (RL) techniques suffer from poor scalability due to state space explosion. Motivated…
Siemens Technology and Services Pvt. Ltd.（西门子技术服务私人有限公司）；International Institute of Information Technology Hyderabad（海得拉巴国际信息技术学院）；Indian Institute of Science Bengaluru（班加罗尔印度科学研究所）
We use Asynchronous Advantage Actor Critic (A3C) for implementing an AI agent in the controllers that optimize flow of traffic across a single intersection and then extend it to multiple intersections by considering a multi-agent setting. We explore three different methodologies to address the multi-agent problem - (1) use of asynchronous property of A3C to control multiple intersections using a single agent (2) utilise self/competitive play among independent agents across multiple intersections and (3) ingest a global reward function among agents to introduce cooperative behavior between inte…
Today's intelligent traffic light control system is based on the current road traffic conditions for traffic regulation. However, these approaches cannot exploit the future traffic information in advance. In this paper, we propose GPlight, a deep reinforcement learning (DRL) algorithm integrated with graph neural network (GNN) , to relieve the traffic congestion for multi-intersection intelligent traffic control system. In GPlight, the graph neural network (GNN) is first used to predict the future short-term traffic flow at the intersections. Then, the results of traffic flow prediction are us…
Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Electrical Engineering, Columbia University（哥伦比亚大学电气工程系）；Nokia Bell Labs（诺基亚贝尔实验室）
Traffic signal control is an important and challenging real-world problem, which aims to minimize the travel time of vehicles by coordinating their movements at the road intersections. Current traffic signal control systems in use still rely heavily on oversimplified information and rule-based methods, although we now have richer data, more computing power and advanced methods to drive the development of intelligent transportation. With the growing interest in intelligent transportation using machine learning methods like reinforcement learning, this survey covers the widely acknowledged trans…
Latest technological improvements increased the quality of transportation. New data-driven approaches bring out a new research direction for all control-based systems, e.g., in transportation, robotics, IoT and power systems. Combining data-driven applications with transportation systems plays a key role in recent transportation applications. In this paper, the latest deep reinforcement learning (RL) based traffic control applications are surveyed. Specifically, traffic signal control (TSC) applications based on (deep) RL, which have been studied extensively in the literature, are discussed in…
Georgia Institute of Technology（佐治亚理工学院）；CableLabs, Inc.（CableLabs公司）
A novel intelligent bandwidth allocation scheme in NG-EPON using reinforcement learning is proposed and demonstrated for latency management. We verify the capability of the proposed scheme under both fixed and dynamic traffic loads scenarios to achieve <1ms average latency. The RL agent demonstrates an efficient intelligent mechanism to manage the latency, which provides a promising IBA solution for the next-generation access network.
GARLIC: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching
City University of Hong Kong（香港城市大学）；Jilin University（吉林大学）；Zhejiang University of Technology（浙江工业大学）；Southern University of Science and Technology（南方科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Alibaba Group（阿里巴巴集团）
Intelligent transport systems have efficiently and effectively proved themselves in settling up the problem of traffic congestion around the world. The multi-agent based transportation system is one of the most important intelligent transport systems, which represents an interaction among the neighbouring vehicles, drivers, roads, infrastructure and vehicles. In this paper, two traffic management models have been created to mitigate congestion and to ensure that emergency vehicles arrive as quickly as possible. A tool-chain SUMO-JADE is employed to create a microscopic simulation symbolizing t…
Connected Autonomous Vehicles will make autonomous intersection management a reality replacing traditional traffic signal control. Autonomous intersection management requires time and speed adjustment of vehicles arriving at an intersection for collision-free passing through the intersection. Due to its computational complexity, this problem has been studied only when vehicle arrival times towards the vicinity of the intersection are known beforehand, which limits the applicability of these solutions for real-time deployment. To solve the real-time autonomous traffic intersection management pr…
Based on game theory and dynamic Level-k model, this paper establishes an intelligent traffic control method for intersections, studies the influence of multi-agent vehicle joint decision-making and group behavior disturbance on system state. The simulation results show that this method has a good performance when there are more vehicles or emergency vehicles have higher priority.
Open radio access network (ORAN) Alliance offers a disaggregated RAN functionality built using open interface specifications between blocks. To efficiently support various competing services, \textit{namely} enhanced mobile broadband (eMBB) and ultra-reliable and low-latency (uRLLC), the ORAN Alliance has introduced a standard approach toward more virtualized, open and intelligent networks. To realize benefits of ORAN in optimizing resource utilization, this paper studies an intelligent traffic steering (TS) scheme within the proposed disaggregated ORAN architecture. For this purpose, we propo…
This paper implements a traffic signal control system by using real-time traffic flow feedback. This system is designed to deal with two-lane intersections. We construct an experiment field similar to the roads and drivers in Taiwan using an autonomous simulation software called Virtual Test Drive (VTD) released by MSC Software. We erect four cameras on the side of the roads to get the image of the intersection, then transfer the image information into traffic flow information. Analyze the traffic information in each lane by using Greenshields traffic flow model. Control the traffic signals by…
In this paper, a unified multi-vehicle formation control framework for Intelligent and Connected Vehicles (ICVs) that can apply to multiple traffic scenarios is proposed. In the one-dimensional scenario, different formation geometries are analyzed and the interlaced structure is mathematically modelized to improve driving safety while making full use of the lane capacity. The assignment problem for vehicles and target positions is solved using Hungarian Algorithm to improve the flexibility of the method in multiple scenarios. In the two-dimensional scenario, an improved virtual platoon method …
SEIDM: A Safe and Efficient Intelligent Driver Model for Autonomous Driving Behavior
Department of Information and Communication Engineering, Tongji University（信息与通信工程系，同济大学）；Department of Electrical and Computer Engineering, Duke University（电气与计算机工程系，杜克大学）
Global Center for Automotive Performance Simulation（全球汽车性能模拟中心）；Honda Research Institute, USA Inc.（美国本田研究院）；Virginia Tech Transportation Institute（弗吉尼亚理工交通研究所）
Department of Civil, Architectural, and Environmental Engineering, University of Texas at Austin（德克萨斯大学奥斯汀分校土木、建筑与环境工程系）；Faculty of Electrical and Computer Engineering, Babol Noshirvani University of Technology（巴布里·纳希里万理工大学电气与计算机工程学院）；Transportation Research Institute (IMOB), UHasselt-Hasselt University（乌赫斯泰特大学交通研究所）；Faculty of Mechanical Engineering, Babol Noshirvani University of Technology（巴布里·纳希里万理工大学机械工程学院）；Department of Computer Engineering, Amirkabir University of Technology（阿姆irkabir技术大学计算机工程系）
