Railway Operation Rescheduling System via Dynamic Simulation and Reinforcement Learning
NEC-AIST AI Cooperative Research Laboratory, National Institute of Advanced Industrial Science and Technology（日本产业技术综合研究所 NEC-AIST 人工智能合作研究实验室）；Data Science Research Laboratories, NEC Corporation（日本电气株式会社数据科学研究实验室）；Graduate School of Information Science and Technology, The University of Tokyo（东京大学信息科学与技术研究生院）
English translated version is placed at first and the original Japanese version follows. 4 pages and 5 figures in the original manuscript. Proceedings of the 28th jointed railway technology symposium (J-RAIL 2021)
The number of railway service disruptions has been increasing owing to intensification of natural disasters. In addition, abrupt changes in social situations such as the COVID-19 pandemic require railway companies to modify the traffic schedule frequently. Therefore, automatic support for optimal scheduling is anticipated. In this study, an automatic railway scheduling system is presented. The system leverages reinforcement learning and a dynamic simulator that can simulate the railway traffic and passenger flow of a whole line. The proposed system enables rapid generation of the traffic schedule of a whole line because the optimization process is conducted in advance as the training. The system is evaluated using an interruption scenario, and the results demonstrate that the system can generate optimized schedules of the whole line in a few minutes.
A Mobile Data-Driven Hierarchical Deep Reinforcement Learning Approach for Real-time Demand-Responsive Railway Rescheduling and Station Overcrowding Mitigation
Chang'an University（长安大学）；University of Leeds（利兹大学）
ZHAW Datalab（苏黎世应用科学大学数据实验室）；ZHAW School of Engineering（苏黎世应用科学大学工程学院）；Swiss Federal Railways（瑞士联邦铁路）
We present preliminary results from our sixth placed entry to the Flatland international competition for train rescheduling, including two improvements for optimized reinforcement learning (RL) training efficiency, and two hypotheses with respect to the prospect of deep RL for complex real-world control tasks: first, that current state of the art policy gradient methods seem inappropriate in the domain of high-consequence environments; second, that learning explicit communication actions (an emerging machine-to-machine language, so to speak) might offer a remedy. These hypotheses need to be co…
We propose a train rescheduling algorithm which applies a standardized feature selection based on pairwise conflicts in order to serve as input for the reinforcement learning framework. We implement an analytical method which identifies and optimally solves every conflict arising between two trains, then we design a corresponding observation space which features the most relevant information considering these conflicts. The data obtained this way then translates to actions in the context of the reinforcement learning framework. We test our preliminary model using the evaluation metrics of the …
Dpto. Ing. en Sistemas de Información (UTN)（国家技术大学 信息系统工程系）
In order to reach higher degrees of flexibility, adaptability and autonomy in manufacturing systems, it is essential to develop new rescheduling methodologies which resort to cognitive capabilities, similar to those found in human beings. Artificial cognition is important for designing planning and control systems that generate and represent knowledge about heuristics for repair-based scheduling. Rescheduling knowledge in the form of decision rules is used to deal with unforeseen events and disturbances reactively in real time, and take advantage of the ability to act interactively with the us…
Dynamic Pricing in High-Speed Railways Using Multi-Agent Reinforcement Learning
Department of Technologies and Information Systems, Universidad de Castilla-La Mancha（卡斯蒂利亚-拉曼查大学技术与信息系统系）；Warwick Manufacturing Group, University of Warwick（沃里克大学制造集团）
Relational Multi-Agent Reinforcement Learning for Dynamic Pricing in High-Speed Railway Markets
Department of Technologies and Information Systems, Universidad de Castilla-La Mancha（卡斯蒂利亚-拉曼恰大学技术与信息系统系）；Warwick Manufacturing Group, University of Warwick（华威大学华威制造集团）
AIcrowd；ETH Zurich（苏黎世联邦理工学院）；FHNW（瑞士西北应用科学与艺术大学）；University of Southern California（南加州大学）；Monash University（莫纳什大学）；HSE University（高等经济大学）；JetBrains Research；Netcetera
The Flatland competition aimed at finding novel approaches to solve the vehicle re-scheduling problem (VRSP). The VRSP is concerned with scheduling trips in traffic networks and the re-scheduling of vehicles when disruptions occur, for example the breakdown of a vehicle. While solving the VRSP in various settings has been an active area in operations research (OR) for decades, the ever-growing complexity of modern railway networks makes dynamic real-time scheduling of traffic virtually impossible. Recently, multi-agent reinforcement learning (MARL) has successfully tackled challenging tasks wh…
Adaptive digital twins for predictive decision-making: Online Bayesian learning of transition dynamics
MOX – Dipartimento di Matematica, Politecnico di Milano（MOX——数学系，米兰理工大学）；Dipartimento di Ingegneria Civile e Ambientale, Politecnico di Milano（土木工程与环境工程系，米兰理工大学）；Mathematics Department, Emory University（埃默里大学数学系）
Efficient automated scheduling of trains remains a major challenge for modern railway systems. The underlying vehicle rescheduling problem (VRSP) has been a major focus of Operations Research (OR) since decades. Traditional approaches use complex simulators to study VRSP, where experimenting with a broad range of novel ideas is time consuming and has a huge computational overhead. In this paper, we introduce a two-dimensional simplified grid environment called "Flatland" that allows for faster experimentation. Flatland does not only reduce the complexity of the full physical simulation, but al…
