Latest technological improvements increased the quality of transportation. New data-driven approaches bring out a new research direction for all control-based systems, e.g., in transportation, robotics, IoT and power systems. Combining data-driven applications with transportation systems plays a key role in recent transportation applications. In this paper, the latest deep reinforcement learning (RL) based traffic control applications are surveyed. Specifically, traffic signal control (TSC) applications based on (deep) RL, which have been studied extensively in the literature, are discussed in detail. Different problem formulations, RL parameters, and simulation environments for TSC are discussed comprehensively. In the literature, there are also several autonomous driving applications studied with deep RL models. Our survey extensively summarizes existing works in this field by categorizing them with respect to application types, control models and studied algorithms. In the end, we discuss the challenges and open questions regarding deep RL-based transportation applications.
We explore the use of deep learning and deep reinforcement learning for optimization problems in transportation. Many transportation system analysis tasks are formulated as an optimization problem - such as optimal control problems in intelligent transportation systems and long term urban planning. Often transportation models used to represent dynamics of a transportation system involve large data sets with complex input-output interactions and are difficult to use in the context of optimization. Use of deep learning metamodels can produce a lower dimensional representation of those relations …
Columbia University（哥伦比亚大学）；Imperial College London（帝国理工学院）；Nokia-Bell Laboratories（诺基亚贝尔实验室）
Intelligent Transportation Systems (ITSs) are envisioned to play a critical role in improving traffic flow and reducing congestion, which is a pervasive issue impacting urban areas around the globe. Rapidly advancing vehicular communication and edge cloud computation technologies provide key enablers for smart traffic management. However, operating viable real-time actuation mechanisms on a practically relevant scale involves formidable challenges, e.g., policy iteration and conventional Reinforcement Learning (RL) techniques suffer from poor scalability due to state space explosion. Motivated…
Intelligent transportation systems (ITSs) are envisioned to be crucial for smart cities, which aims at improving traffic flow to improve the life quality of urban residents and reducing congestion to improve the efficiency of commuting. However, several challenges need to be resolved before such systems can be deployed, for example, conventional solutions for Markov decision process (MDP) and single-agent Reinforcement Learning (RL) algorithms suffer from poor scalability, and multi-agent systems suffer from poor communication and coordination. In this paper, we explore the potential of mutual…
In this paper we propose a Deep Reinforcement Learning approach to solve a multimodal transportation planning problem, in which containers must be assigned to a truck or to trains that will transport them to their destination. While traditional planning methods work "offline" (i.e., they take decisions for a batch of containers before the transportation starts), the proposed approach is "online", in that it can take decisions for individual containers, while transportation is being executed. Planning transportation online helps to effectively respond to unforeseen events that may affect the or…
School of Industrial Engineering, Purdue University（普渡大学工业工程学院）
The growth in online goods delivery is causing a dramatic surge in urban vehicle traffic from last-mile deliveries. On the other hand, ride-sharing has been on the rise with the success of ride-sharing platforms and increased research on using autonomous vehicle technologies for routing and matching. The future of urban mobility for passengers and goods relies on leveraging new methods that minimize operational costs and environmental footprints of transportation systems. This paper considers combining passenger transportation with goods delivery to improve vehicle-based transportation. Even t…
University of New South Wales（新南威尔士大学）；University of Sydney（悉尼大学）；Technische Universität Dresden（德累斯顿工业大学）；The Hong Kong Polytechnic University（香港理工大学）
Transportation is the backbone of the economy and urban development. Improving the efficiency, sustainability, resilience, and intelligence of transportation systems is critical and also challenging. The constantly changing traffic conditions, the uncertain influence of external factors (e.g., weather, accidents), and the interactions among multiple travel modes and multi-type flows result in the dynamic and stochastic natures of transportation systems. The planning, operation, and control of transportation systems require flexible and adaptable strategies in order to deal with uncertainty, no…
Western University（西安大略大学）；ISSAE-Cnam Liban（黎巴嫩国立应用科学学院-法国国立工艺学院）；CNAM Paris（巴黎法国国立工艺学院）
The growing deployment efforts of 5G networks globally has led to the acceleration of the businesses/services' digital transformation. This growth has led to the need for new communication technologies that will promote this transformation. 6G is being proposed as the set of technologies and architectures that will achieve this target. Among the main use cases that have emerged for 5G networks and will continue to play a pivotal role in 6G networks is that of Intelligent Transportation Systems (ITSs). With all the projected benefits of developing and deploying efficient and effective ITSs come…
SIGMA: Symmetry-aware, Intelligent, Geometric, Multi-faced Adaptive Traffic Control for Reliable Intelligent Transportation Systems
Indian Statistical Institute（印度统计研究所）；Sorbonne University Abu Dhabi（阿布扎比索邦大学）；Sorbonne Center for Artificial Intelligence（索邦人工智能中心）；SQC & OR（统计质量控制与运筹学研究室）
Deep reinforcement learning (DRL) is an emerging methodology that is transforming the way many complicated transportation decision-making problems are tackled. Researchers have been increasingly turning to this powerful learning-based methodology to solve challenging problems across transportation fields. While many promising applications have been reported in the literature, there remains a lack of comprehensive synthesis of the many DRL algorithms and their uses and adaptations. The objective of this paper is to fill this gap by conducting a comprehensive, synthesized review of DRL applicati…
Multi-agent deep reinforcement learning with centralized training and decentralized execution for transportation infrastructure management
organization= Department of Civil；Environmental Engineering , addressline= The Pennsylvania State University , city= University Park , postcode= 16802 , state= PA , country= USA；organization= Faculty of Architecture \& the Built Environment , addressline= Delft University of Technology , postcode= 2628 BL , state= Delft , country= The Netherlands
Georgia Institute of Technology（佐治亚理工学院）；CableLabs, Inc.（CableLabs公司）
A novel intelligent bandwidth allocation scheme in NG-EPON using reinforcement learning is proposed and demonstrated for latency management. We verify the capability of the proposed scheme under both fixed and dynamic traffic loads scenarios to achieve <1ms average latency. The RL agent demonstrates an efficient intelligent mechanism to manage the latency, which provides a promising IBA solution for the next-generation access network.
School of Computer Science, Guangzhou University（广州大学计算机学院）；Qingdao University of Technology（青岛理工大学）
In this paper, we investigate mobile edge computing (MEC) networks for intelligent internet of things (IoT), where multiple users have some computational tasks assisted by multiple computational access points (CAPs). By offloading some tasks to the CAPs, the system performance can be improved through reducing the latency and energy consumption, which are the two important metrics of interest in the MEC networks. We devise the system by proposing the offloading strategy intelligently through the deep reinforcement learning algorithm. In this algorithm, Deep Q-Network is used to automatically le…
Radio Resource Management and Path Planning in Intelligent Transportation Systems via Reinforcement Learning for Environmental Sustainability
Tarbiat Modares University（塔比亚特·莫达雷斯大学）；Doha University for Science and Technology（多哈科技大学）；TU Braunschweig（布伦瑞克工业大学）；Institute of Communications Technology（通信技术研究所）
Traffic signal control is an important and challenging real-world problem, which aims to minimize the travel time of vehicles by coordinating their movements at the road intersections. Current traffic signal control systems in use still rely heavily on oversimplified information and rule-based methods, although we now have richer data, more computing power and advanced methods to drive the development of intelligent transportation. With the growing interest in intelligent transportation using machine learning methods like reinforcement learning, this survey covers the widely acknowledged trans…
Joint Task Offloading, Inference Optimization and UAV Trajectory Planning for Generative AI Empowered Intelligent Transportation Digital Twin
Guangxi University Key Laboratory of Intelligent Networking and Scenario System (School of Information and Communication, Guilin University of Electronic Technology)（广西大学智能组网与场景系统重点实验室（桂林电子科技大学信息与通信学院））；Research Institute of Highway Ministry of Transport（交通运输部公路科学研究院）；School of Automation, Guangdong University of Technology（广东工业大学自动化学院）；School of Architecture and Transportation Engineering, Guilin University of Electronic Technology（桂林电子科技大学建筑与交通工程学院）
An Overview of Machine Learning-Enabled Optimization for Reconfigurable Intelligent Surfaces-Aided 6G Networks: From Reinforcement Learning to Large Language Models
School of Computer Science, McGill University（麦吉尔大学计算机科学学院）
Strategizing Equitable Transit Evacuations: A Data-Driven Reinforcement Learning Approach
School of Sustainable Engineering and the Built Environment, Arizona State University（亚利桑那州立大学可持续工程与建筑环境学院）；University of California, Berkeley（加州大学伯克利分校）
Deep Reinforcement Learning for Traffic Light Control in Intelligent Transportation Systems
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Department of Electrical Engineering, Columbia University（哥伦比亚大学电气工程系）；Nokia Bell Labs（诺基亚贝尔实验室）
Storage systems for cloud computing merge a large number of commodity computers into a single large storage pool. It provides high-performance storage over an unreliable, and dynamic network at a lower cost than purchasing and maintaining large mainframe. In this paper, we examine whether it is feasible to apply Reinforcement Learning(RL) to system domain problems. Our experiments show that the RL model is comparable, even outperform other heuristics for block management problem. However, our experiments are limited in terms of scalability and fidelity. Even though our formulation is not very …
Intelligent Traffic Light Control System (ITLCS) is a typical Multi-Agent System (MAS), which comprises multiple roads and traffic lights.Constructing a model of MAS for ITLCS is the basis to alleviate traffic congestion. Existing approaches of MAS are largely based on Multi-Agent Deep Reinforcement Learning (MADRL). Although the Deep Neural Network (DNN) of MABRL is effective, the training time is long, and the parameters are difficult to trace. Recently, Broad Learning Systems (BLS) provided a selective way for learning in the deep neural networks by a flat network. Moreover, Broad Reinforce…
LLM-based Personalized Portfolio Recommender: Integrating Large Language Models and Reinforcement Learning for Intelligent Investment Strategy Optimization
University of Electronic Science and Technology of China（电子科技大学）；Ross School of Business, University of Michigan（密歇根大学罗斯商学院）；School of Humanities and Sciences, Stanford University（斯坦福大学人文与科学学院）
Queen’s University Belfast（贝尔法斯特女王大学）；Duy Tan University（阮攸大学）；Memorial University（纪念大学）
In this paper, we propose a deep reinforcement learning (DRL) approach for solving the optimisation problem of the network's sum-rate in device-to-device (D2D) communications supported by an intelligent reflecting surface (IRS). The IRS is deployed to mitigate the interference and enhance the signal between the D2D transmitter and the associated D2D receiver. Our objective is to jointly optimise the transmit power at the D2D transmitter and the phase shift matrix at the IRS to maximise the network sum-rate. We formulate a Markov decision process and then propose the proximal policy optimisatio…
The University of Iowa（爱荷华大学）；National Advanced Driving Simulator, The University of Iowa（爱荷华大学国家高级驾驶模拟器）；Iowa Technology Institute, The University of Iowa（爱荷华大学爱荷华技术研究所）
Intelligent traffic lights in smart cities can optimally reduce traffic congestion. In this study, we employ reinforcement learning to train the control agent of a traffic light on a simulator of urban mobility. As a difference from existing works, a policy-based deep reinforcement learning method, Proximal Policy Optimization (PPO), is utilized other than value-based methods such as Deep Q Network (DQN) and Double DQN (DDQN). At first, the obtained optimal policy from PPO is compared to those from DQN and DDQN. It is found that the policy from PPO performs better than the others. Next, instea…
Intelligent transport systems have efficiently and effectively proved themselves in settling up the problem of traffic congestion around the world. The multi-agent based transportation system is one of the most important intelligent transport systems, which represents an interaction among the neighbouring vehicles, drivers, roads, infrastructure and vehicles. In this paper, two traffic management models have been created to mitigate congestion and to ensure that emergency vehicles arrive as quickly as possible. A tool-chain SUMO-JADE is employed to create a microscopic simulation symbolizing t…
Baidu Research（百度研究院）；Baidu Inc.（百度公司）；University of Central Florida（中佛罗里达大学）；Rutgers University（罗格斯大学）
Public transportation plays a critical role in people's daily life. It has been proven that public transportation is more environmentally sustainable, efficient, and economical than any other forms of travel. However, due to the increasing expansion of transportation networks and more complex travel situations, people are having difficulties in efficiently finding the most preferred route from one place to another through public transportation systems. To this end, in this paper, we present Polestar, a data-driven engine for intelligent and efficient public transportation routing. Specifically…
Shahrood University of Technology（沙赫鲁德理工大学）；Louisiana State University（路易斯安那州立大学）；Ton Duc Thang University（同塔大学）；Oxford Brookes University（牛津布鲁克斯大学）；Bauhaus University Weimar（魏玛包豪斯大学）；Queensland University of Technology（昆士兰科技大学）；Duy Tan University（岘港大学（越南东塔大学））；J. Selye University（塞莱大学）；Obuda University（奥布达大学）
Prediction models in mobility and transportation maintenance systems have been dramatically improved through using machine learning methods. This paper proposes novel machine learning models for intelligent road inspection. The traditional road inspection systems based on the pavement condition index (PCI) are often associated with the critical safety, energy and cost issues. Alternatively, the proposed models utilize surface deflection data from falling weight deflectometer (FWD) tests to predict the PCI. Machine learning methods are the single multi-layer perceptron (MLP) and radial basis fu…
Architecting Digital Twins for Intelligent Transportation Systems
Software Engineering Research Centre, IIIT Hyderabad（印度信息技术与管理大学海得拉巴分校软件工程研究中心）；Computer Systems Group, IIIT Hyderabad（印度信息技术与管理大学海得拉巴分校计算机系统组）；Faculty of Science and Technology, Middlesex University London（伦敦密德萨斯大学科学与技术学院）
Missing data is a challenge in many applications, including intelligent transportation systems (ITS). In this paper, we study traffic speed and travel time estimations in ITS, where portions of the collected data are missing due to sensor instability and communication errors at collection points. These practical issues can be remediated by missing data analysis, which are mainly categorized as either statistical or machine learning(ML)-based approaches. Statistical methods require the prior probability distribution of the data which is unknown in our application. Therefore, we focus on an ML-b…
With the incoming introduction of 5G networks and the advancement in technologies, such as Network Function Virtualization and Software Defined Networking, new and emerging networking technologies and use cases are taking shape. One such technology is the Internet of Vehicles (IoV), which describes an interconnected system of vehicles and infrastructure. Coupled with recent developments in artificial intelligence and machine learning, the IoV is transformed into an Intelligent Transportation System (ITS). There are, however, several operational considerations that hinder the adoption of ITS sy…
Multimodal LLM for Intelligent Transportation Systems
Drexel University（德雷塞尔大学）；Purdue University（普渡大学）；Indian Institute of Science（印度科学学院）；Stanford University（斯坦福大学）；Temsa Research & Development Center（Temsa研发中心）
Intelligent Transportation Systems (ITS) are much correlated with data science mechanisms. Among the different correlation branches, this paper focuses on the neural network learning models. Some of the considered models are shallow and they get some user-defined features and learn the relationship, while deep models extract the necessary features before learning by themselves. Both of these paradigms are utilized in the recent intelligent transportation systems (ITS) to support decision-making by the aid of different operations such as frequent patterns mining, regression, clustering, and cla…
Cooperative Intelligent Transport System (C-ITS) is a promising network to provide safety, efficiency, sustainability, and comfortable services for automated vehicles and road infrastructures by taking advantages from participants. However, the components of C-ITS usually generate large amounts of data, which makes it difficult to explore data science. Currently, federated learning has been proposed as an appealing approach to allow users to cooperatively reap the benefits from trained participants. Therefore, in this paper, we propose a novel Semi-asynchronous Hierarchical Federated Learning …
Air pollution and carbon emissions caused by modern transportation are closely related to global climate change. With the help of next-generation information technology such as Internet of Things (IoT) and Artificial Intelligence (AI), accurate traffic flow prediction can effectively solve problems such as traffic congestion and mitigate environmental pollution and climate change. It further promotes the development of Intelligent Transportation Systems (ITS) and smart cities. However, the strong spatial and temporal correlation of traffic data makes the task of accurate traffic forecasting a …
V2X-Boosted Federated Learning for Cooperative Intelligent Transportation Systems with Contextual Client Selection
Fraunhofer Institute for Transportation and Infrastructure Systems IVI（弗劳恩霍夫交通与基础设施系统研究所IVI）；Technical University of Munich（慕尼黑工业大学）；Sony AI（索尼人工智能研究所）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心（DFKI））；Technische Hochschule Ingolstadt（英戈尔施塔特应用技术大学）
Queen Mary University of London（伦敦玛丽女王大学）；Fotech Group（Fotech集团）
Intelligent transport systems (ITS) are pivotal in the development of sustainable and green urban living. ITS is data-driven and enabled by the profusion of sensors ranging from pneumatic tubes to smart cameras. This work explores a novel data source based on optical fibre-based distributed acoustic sensors (DAS) for traffic analysis. Detecting the type of vehicle and estimating the occupancy of vehicles are prime concerns in ITS. The first is motivated by the need for tracking, controlling, and forecasting traffic flow. The second targets the regulation of high occupancy vehicle lanes in an a…
TECNALIA, Basque Research & Technology Alliance (BRTA)（泰克纳利亚，巴斯克研究与技术联盟（BRTA））；University of Las Palmas de Gran Canaria（拉斯帕尔马斯大学）；National Technical University of Athens（雅典国家技术大学）；University of the Basque Country (UPV/EHU)（巴斯克大学（UPV/EHU））
Advances in Data Science permeate every field of Transportation Science and Engineering, resulting in developments in the transportation sector that {are} data-driven. Nowadays, Intelligent Transportation Systems (ITS) could be arguably approached as a ``story'' intensively producing and consuming large amounts of data. A~diversity of sensing devices densely spread over the infrastructure, vehicles or the travelers' personal devices act as sources of data flows that are eventually fed {into} software running on automatic devices, actuators or control systems producing, in~turn, complex informa…
Kyung Hee University（庆熙大学）；Mid Sweden University（中瑞典大学）；BRAC University（孟加拉国布拉克大学）
This paper presents a cognitive behavioral-based driver mood repairment platform in intelligent transportation cyber-physical systems (IT-CPS) for road safety. In particular, we propose a driving safety platform for distracted drivers, namely \emph{drive safe}, in IT-CPS. The proposed platform recognizes the distracting activities of the drivers as well as their emotions for mood repair. Further, we develop a prototype of the proposed drive safe platform to establish proof-of-concept (PoC) for the road safety in IT-CPS. In the developed driving safety platform, we employ five AI and statistica…
Palo Alto Research Center（帕洛阿尔托研究中心）；Virginia Tech Transportation Institute（弗吉尼亚理工交通研究所）
Our research aims at developing intelligent systems to reduce the transportation-related energy expenditure of a large city by influencing individual behavior. We introduce COPTER - an intelligent travel assistant that evaluates multi-modal travel alternatives to find a plan that is acceptable to a person given their context and preferences. We propose a formulation for acceptable planning that brings together ideas from AI, machine learning, and economics. This formulation has been incorporated in COPTER that produces acceptable plans in real-time. We adopt a novel empirical evaluation framew…
School of Computer Science and Technology（计算机科学与技术学院）；Taiyuan Normal University（太原师范学院）；Institute of Computer Science（计算机科学研究所）；University of Göttingen（哥廷根大学）；James Watt School of Engineering（詹姆斯·瓦特工程学院）；University of Glasgow（格拉斯哥大学）；Laboratory of Intelligent Home Appliances（智能家电实验室）；College of Science and Technology（科学与技术学院）；Ningbo University（宁波大学）
Federated Learning in Intelligent Transportation Systems: Recent Applications and Open Problems
Nanjing University of Science and Technology（南京理工大学）；Data61, CSIRO（澳大利亚联邦科学与工业研究组织Data61）；Purdue University（普渡大学）；Jinan University（暨南大学）；University of Houston（休斯顿大学）
ETH Zürich（苏黎世联邦理工学院）；Prescient Design, Genentech | Roche（罗氏旗下基因泰克Prescient Design）；Swiss Data Science Center（瑞士数据科学中心）
Artificial intelligence (AI) is revolutionizing many areas of our lives, leading a new era of technological advancement. Particularly, the transportation sector would benefit from the progress in AI and advance the development of intelligent transportation systems. Building intelligent transportation systems requires an intricate combination of artificial intelligence and mobility analysis. The past few years have seen rapid development in transportation applications using advanced deep neural networks. However, such deep neural networks are difficult to interpret and lack robustness, which sl…
