We propose a novel approach to optimize fleet management by combining multi-agent reinforcement learning with graph neural network. To provide ride-hailing service, one needs to optimize dynamic resources and demands over spatial domain. While the spatial structure was previously approximated with a regular grid, our approach represents the road network with a graph, which better reflects the underlying geometric structure. Dynamic resource allocation is formulated as multi-agent reinforcement learning, whose action-value function (Q function) is approximated with graph neural networks. We use stochastic policy update rule over the graph with deep Q-networks (DQN), and achieve superior results over the greedy policy update. We design a realistic simulator that emulates the empirical taxi call data, and confirm the effectiveness of the proposed model under various conditions.
Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究所）；National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）
The future of mobility-as-a-Service (Maas)should embrace an integrated system of ride-hailing, street-hailing and ride-sharing with optimised intelligent vehicle routing in response to a real-time, stochastic demand pattern. We aim to optimise routing policies for a large fleet of vehicles for street-hailing services, given a stochastic demand pattern in small to medium-sized road networks. A model-based dispatch algorithm, a high performance model-free reinforcement learning based algorithm and a novel hybrid algorithm combining the benefits of both the top-down approach and the model-free re…
Nowadays, autonomous taxis become a highly promising transportation mode, which helps relieve traffic congestion and avoid road accidents. However, it hinders the wide implementation of this service that traditional models fail to efficiently allocate the available fleet to deal with the imbalance of supply (autonomous taxis) and demand (trips), the poor cooperation of taxis, hardly satisfied resource constraints, and on-line platform's requirements. To figure out such urgent problems from a global and more farsighted view, we employ a Constrained Multi-agent Markov Decision Processes (CMMDP) …
Optimizing Electric Bus Charging Scheduling with Uncertainties Using Hierarchical Deep Reinforcement Learning
School of Engineering, University of Guelph（圭尔夫大学工程学院）；EthicalAI（伦理AI）；College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算与数据科学学院）
Traditional traffic optimization solutions assume that the graph structure of road networks is static, missing opportunities for further traffic flow optimization. We are interested in optimizing traffic flows as a new type of graph-based problem, where the graph structure of a road network can adapt to traffic conditions in real time. In particular, we focus on the dynamic configuration of traffic-lane directions, which can help balance the usage of traffic lanes in opposite directions. The rise of connected autonomous vehicles offers an opportunity to apply this type of dynamic traffic optim…
With the continuous growth of the global economy and markets, resource imbalance has risen to be one of the central issues in real logistic scenarios. In marine transportation, this trade imbalance leads to Empty Container Repositioning (ECR) problems. Once the freight has been delivered from an exporting country to an importing one, the laden will turn into empty containers that need to be repositioned to satisfy new goods requests in exporting countries. In such problems, the performance that any cooperative repositioning policy can achieve strictly depends on the routes that vessels will fo…
ROAD: Responsibility-Oriented Reward Design for Reinforcement Learning in Autonomous Driving
Global Institute of Future Technology（未来技术全球研究院）；Shanghai Jiao Tong University（上海交通大学）；College of Humanities（人文学院）；Department of Automation（自动化系）；School of Aeronautics and Astronautics（航空宇航学院）
BMW Group（宝马集团）；University of the Federal Armed Forces Munich（慕尼黑联邦国防军大学）
Most traffic state forecast algorithms when applied to urban road networks consider only the links in close proximity to the target location. However, for longer-term forecasts also the traffic state of more distant links or regions of the network are expected to provide valuable information for a data-driven algorithm. This paper studies these expectations of using a network clustering algorithm and one year of Floating Car (FCD) collected by a large fleet of vehicles. First, a clustering algorithm is applied to the data in order to extract congestion-prone regions in the Munich city network.…
Modern vehicle fleets, e.g., for ridesharing platforms and taxi companies, can reduce passengers' waiting times by proactively dispatching vehicles to locations where pickup requests are anticipated in the future. Yet it is unclear how to best do this: optimal dispatching requires optimizing over several sources of uncertainty, including vehicles' travel times to their dispatched locations, as well as coordinating between vehicles so that they do not attempt to pick up the same passenger. While prior works have developed models for this uncertainty and used them to optimize dispatch policies, …
Shanghai Jiao Tong University（上海交通大学）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；the Hong Kong University of Science and Technology（香港科技大学）
Semantic road region segmentation is a high-level task, which paves the way towards road scene understanding. This paper presents a residual network trained for semantic road segmentation. Firstly, we represent the projections of road disparities in the v-disparity map as a linear model, which can be estimated by optimizing the v-disparity map using dynamic programming. This linear model is then utilized to reduce the redundant information in the left and right road images. The right image is also transformed into the left perspective view, which greatly enhances the road surface similarity be…
TensorCell；Faculty of Mathematics, Informatics and Mechanics（数学、信息学与力学学院）；University of Warsaw（华沙大学）
We present a method for approximating outcomes of road traffic simulations using BERT-based models, which may find applications in, e.g., optimizing traffic signal settings, especially with the presence of autonomous and connected vehicles. The experiments were conducted on a dataset generated using the Traffic Simulation Framework software runs on a realistic road network. The BERT-based models were compared with 4 other types of machine learning models (LightGBM, fully connected neural networks and 2 types of graph neural networks) and gave the best results in terms of all the considered met…
