Dynamic dispatching aims to smartly allocate the right resources to the right place at the right time. Dynamic dispatching is one of the core problems for operations optimization in the mining industry. Theoretically, deep reinforcement learning (RL) should be a natural fit to solve this problem. However, the industry relies on heuristics or even human intuitions, which are often short-sighted and sub-optimal solutions. In this paper, we review the main challenges in using deep RL to address the dynamic dispatching problem in the mining industry.
Beijing University of Posts and Telecommunications（北京邮电大学）；China Electric Power Research Institute（中国电力科学研究院）
The expansion of renewable energy could help realizing the goals of peaking carbon dioxide emissions and carbon neutralization. Some existing grid dispatching methods integrating short-term renewable energy prediction and reinforcement learning (RL) have been proved to alleviate the adverse impact of energy fluctuations risk. However, these methods omit the long-term output prediction, which leads to stability and security problems on the optimal power flow. This paper proposes a confidence estimation Transformer for long-term renewable energy forecasting in reinforcement learning-based power …
Reinforcement Learning with Curriculum-inspired Adaptive Direct Policy Guidance for Truck Dispatching
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Hunan University（湖南大学）；State University of New York（纽约州立大学）；University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Agency for Science Technology and Research (A*STAR)（科学、技术与研究局（A*STAR））；Institute for Infocomm Research（资讯通信研究院）
As a new generation of Public Bicycle-sharing Systems (PBS), the dockless PBS (DL-PBS) is an important application of cyber-physical systems and intelligent transportation. How to use AI to provide efficient bicycle dispatching solutions based on dynamic bicycle rental demand is an essential issue for DL-PBS. In this paper, we propose a dynamic bicycle dispatching algorithm based on multi-objective reinforcement learning (MORL-BD) to provide the optimal bicycle dispatching solution for DL-PBS. We model the DL-PBS system from the perspective of CPS and use deep learning to predict the layout of…
Order dispatching and driver repositioning (also known as fleet management) in the face of spatially and temporally varying supply and demand are central to a ride-sharing platform marketplace. Hand-crafting heuristic solutions that account for the dynamics in these resource allocation problems is difficult, and may be better handled by an end-to-end machine learning method. Previous works have explored machine learning methods to the problem from a high-level perspective, where the learning method is responsible for either repositioning the drivers or dispatching orders, and as a further simp…
Efficient dispatching rule in manufacturing industry is key to ensure product on-time delivery and minimum past-due and inventory cost. Manufacturing, especially in the developed world, is moving towards on-demand manufacturing meaning a high mix, low volume product mix. This requires efficient dispatching that can work in dynamic and stochastic environments, meaning it allows for quick response to new orders received and can work over a disparate set of shop floor settings. In this paper we address this problem of dispatching in manufacturing. Using reinforcement learning (RL), we propose a n…
Hitachi America Ltd.（日立美国有限公司）；Stevens Institute of Technology（史蒂文斯理工学院）
Dynamic dispatching is one of the core problems for operation optimization in traditional industries such as mining, as it is about how to smartly allocate the right resources to the right place at the right time. Conventionally, the industry relies on heuristics or even human intuitions which are often short-sighted and sub-optimal solutions. Leveraging the power of AI and Internet of Things (IoT), data-driven automation is reshaping this area. However, facing its own challenges such as large-scale and heterogenous trucks running in a highly dynamic environment, it can barely adopt methods de…
GARLIC: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching
City University of Hong Kong（香港城市大学）；Jilin University（吉林大学）；Zhejiang University of Technology（浙江工业大学）；Southern University of Science and Technology（南方科技大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Alibaba Group（阿里巴巴集团）
In this paper, we study a courier dispatching problem (CDP) raised from an online pickup-service platform of Alibaba. The CDP aims to assign a set of couriers to serve pickup requests with stochastic spatial and temporal arrival rate among urban regions. The objective is to maximize the revenue of served requests given a limited number of couriers over a period of time. Many online algorithms such as dynamic matching and vehicle routing strategy from existing literature could be applied to tackle this problem. However, these methods rely on appropriately predefined optimization objectives at e…
Didi Chuxing（滴滴出行）；Washington State University（华盛顿州立大学）；Northwestern University（西北大学）
Recent works on ride-sharing order dispatching have highlighted the importance of taking into account both the spatial and temporal dynamics in the dispatching process for improving the transportation system efficiency. At the same time, deep reinforcement learning has advanced to the point where it achieves superhuman performance in a number of fields. In this work, we propose a deep reinforcement learning based solution for order dispatching and we conduct large scale online A/B tests on DiDi's ride-dispatching platform to show that the proposed method achieves significant improvement on bot…
North Carolina State University（北卡罗来纳州立大学）；London School of Economics and Political Science（伦敦政治经济学院）
Order dispatch is one of the central problems to ride-sharing platforms. Recently, value-based reinforcement learning algorithms have shown promising performance on this problem. However, in real-world applications, the non-stationarity of the demand-supply system poses challenges to re-utilizing data generated in different time periods to learn the value function. In this work, motivated by the fact that the relative relationship between the values of some states is largely stable across various environments, we propose a pattern transfer learning framework for value-based reinforcement learn…
Shanghai Jiao Tong University（上海交通大学）；DiDi AI Labs（滴滴人工智能实验室）；DiDi Research（滴滴研究院）
Improving the efficiency of dispatching orders to vehicles is a research hotspot in online ride-hailing systems. Most of the existing solutions for order-dispatching are centralized controlling, which require to consider all possible matches between available orders and vehicles. For large-scale ride-sharing platforms, there are thousands of vehicles and orders to be matched at every second which is of very high computational cost. In this paper, we propose a decentralized execution order-dispatching method based on multi-agent reinforcement learning to address the large-scale order-dispatchin…
Integrated Order Dispatching and Routing for Last-Mile Pickup via Deep Reinforcement Learning
College of Management and Economics, Tianjin University（天津大学管理与经济学部）；Laboratory of Computation and Analytics of Complex Management Systems (CACMS), Tianjin University（天津大学复杂管理系统计算与分析实验室）
In this study, a real-time dispatching algorithm based on reinforcement learning is proposed and for the first time, is deployed in large scale. Current dispatching methods in ridehailing platforms are dominantly based on myopic or rule-based non-myopic approaches. Reinforcement learning enables dispatching policies that are informed of historical data and able to employ the learned information to optimize returns of expected future trajectories. Previous studies in this field yielded promising results, yet have left room for further improvements in terms of performance gain, self-dependency, …
Significant development of ride-sharing services presents a plethora of opportunities to transform urban mobility by providing personalized and convenient transportation while ensuring efficiency of large-scale ride pooling. However, a core problem for such services is route planning for each driver to fulfill the dynamically arriving requests while satisfying given constraints. Current models are mostly limited to static routes with only two rides per vehicle (optimally) or three (with heuristics). In this paper, we present a dynamic, demand aware, and pricing-based vehicle-passenger matching…
Scalability of Reinforcement Learning Methods for Dispatching in Semiconductor Frontend Fabs: A Comparison of Open-Source Models with Real Industry Datasets
Infineon Technologies AG（英飞凌科技有限公司）；University of Klagenfurt（克雷格福尔特大学）；Technical University Munich（慕尼黑技术大学）；University of Padua（帕多瓦大学）；D-SIMLAB Technologies Pte Ltd（D-SIMLAB技术私人有限公司）
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
This paper introduces a Reinforcement Learning approach to better generalize heuristic dispatching rules on the Job-shop Scheduling Problem (JSP). Current models on the JSP do not focus on generalization, although, as we show in this work, this is key to learning better heuristics on the problem. A well-known technique to improve generalization is to learn on increasingly complex instances using Curriculum Learning (CL). However, as many works in the literature indicate, this technique might suffer from catastrophic forgetting when transferring the learned skills between different problem size…
The increasing penetration of renewable generations, along with the deregulation and marketization of power industry, promotes the transformation of power market operation paradigms. The optimal bidding strategy and dispatching methodology under these new paradigms are prioritized concerns for both market participants and power system operators, with obstacles of uncertain characteristics, computational efficiency, as well as requirements of hyperopic decision-making. To tackle these problems, the Reinforcement Learning (RL), as an emerging machine learning technique with advantages compared w…
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；School of Intelligent Systems Science and Engineering, Jinan University（暨南大学智能系统科学与工程学院）
The dynamic job-shop scheduling problem (DJSP) is a class of scheduling tasks that specifically consider the inherent uncertainties such as changing order requirements and possible machine breakdown in realistic smart manufacturing settings. Since traditional methods cannot dynamically generate effective scheduling strategies in face of the disturbance of environments, we formulate the DJSP as a Markov decision process (MDP) to be tackled by reinforcement learning (RL). For this purpose, we propose a flexible hybrid framework that takes disjunctive graphs as states and a set of general dispatc…
Preference-Agile Multi-Objective Optimization for Real-time Vehicle Dispatching
School of Computer Science, University of Nottingham Ningbo China（宁波诺丁汉大学计算机科学学院）；School of Computer Science, University of Nottingham（诺丁汉大学计算机科学学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Shibaura Institute of Technology（芝浦工业大学）；Tokyo Institute of Technology（东京工业大学）；Kyoto University（京都大学）；Rutgers University（罗格斯大学）
This paper presents the watch-from-sky framework, where multiple unmanned aerial vehicles (UAVs) play four roles, i.e., sensing, data forwarding, computing, and patrolling, for predictive police surveillance. Our framework is promising for crime deterrence because UAVs are useful for collecting and distributing data and have high mobility. Our framework relies on machine learning (ML) technology for controlling and dispatching UAVs and predicting crimes. This paper compares the conceptual model of our framework against the literature. It also reports a simulation of UAV dispatching using reinf…
LLM-ODDR: A Large Language Model Framework for Joint Order Dispatching and Driver Repositioning
Thrust of Artificial Intelligence, The Hong Kong University of Science and Technology (Guangzhou)（人工智能前沿技术 thrust，香港科学与技术大学（广州））；Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University（航空与航空工程系，香港理工大学）；Research Center for Low Altitude Economy, The Hong Kong Polytechnic University（低空经济研究中心，香港理工大学）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（土木与环境工程系，香港科学与技术大学）
Deep Reinforcement Learning for Flexible Job Shop Scheduling with Random Job Arrivals
Centre for Artificial Intelligence, Zurich University of Applied Sciences（人工智能中心，苏黎世应用科学大学）；Inspire AG（Inspire公司）；Automatic Control Laboratory, ETH Zurich（自动控制实验室，苏黎世联邦理工学院）
Nanyang Technological University（南洋理工大学）；Institute of Marine Science and Technology, Shandong University（山东大学海洋科学与技术研究院）；National University of Singapore（新加坡国立大学）；Singapore Institute of Manufacturing Technology, A*STAR（新加坡科技研究局制造技术研究院）
Priority dispatching rule (PDR) is widely used for solving real-world Job-shop scheduling problem (JSSP). However, the design of effective PDRs is a tedious task, requiring a myriad of specialized knowledge and often delivering limited performance. In this paper, we propose to automatically learn PDRs via an end-to-end deep reinforcement learning agent. We exploit the disjunctive graph representation of JSSP, and propose a Graph Neural Network based scheme to embed the states encountered during solving. The resulting policy network is size-agnostic, effectively enabling generalization on large…
This paper introduces an adaptive model-free deep reinforcement approach that can recognize and adapt to the diurnal patterns in the ride-sharing environment with car-pooling. Deep Reinforcement Learning (RL) suffers from catastrophic forgetting due to being agnostic to the timescale of changes in the distribution of experiences. Although RL algorithms are guaranteed to converge to optimal policies in Markov decision processes (MDPs), this only holds in the presence of static environments. However, this assumption is very restrictive. In many real-world problems like ride-sharing, traffic cont…
University of Klagenfurt（克拉根福大学）；Graz University of Technology（格拉茨技术大学）；Infineon Technologies（英飞凌科技）
Semiconductor manufacturing is a notoriously complex and costly multi-step process involving a long sequence of operations on expensive and quantity-limited equipment. Recent chip shortages and their impacts have highlighted the importance of semiconductors in the global supply chains and how reliant on those our daily lives are. Due to the investment cost, environmental impact, and time scale needed to build new factories, it is difficult to ramp up production when demand spikes. This work introduces a method to successfully learn to schedule a semiconductor manufacturing facility more effici…
Scaliro GmbH（Scaliro有限公司）；Rosenheim University of Applied Sciences（罗森海姆应用科学大学）
The scheduling of production resources (such as associating jobs to machines) plays a vital role for the manufacturing industry not only for saving energy but also for increasing the overall efficiency. Among the different job scheduling problems, the JSSP is addressed in this work. JSSP falls into the category of NP-hard COP, in which solving the problem through exhaustive search becomes unfeasible. Simple heuristics such as FIFO, LPT and metaheuristics such as Taboo search are often adopted to solve the problem by truncating the search space. The viability of the methods becomes inefficient …
University of Oxford（牛津大学）；Imperial College London（帝国理工学院）
Scheduling plays an important role in automated production. Its impact can be found in various fields such as the manufacturing industry, the service industry and the technology industry. A scheduling problem (NP-hard) is a task of finding a sequence of job assignments on a given set of machines with the goal of optimizing the objective defined. Methods such as Operation Research, Dispatching Rules, and Combinatorial Optimization have been applied to scheduling problems but no solution guarantees to find the optimal solution. The recent development of Reinforcement Learning has shown success i…
Applying Ensemble Models based on Graph Neural Network and Reinforcement Learning for Wind Power Forecasting
Northwestern University（西北大学）；Lanzhou University（兰州大学）；South China University of Technology（华南理工大学）；Xi’an Jiaotong University（西安交通大学）；Cloud Gansu Technology Co., Ltd（甘肃云科技有限公司）；Deer AI (Gansu) Technology Co., Ltd（迪尔人工智能（甘肃）科技有限公司）
Linear-Quadratic Mean-Field Reinforcement Learning: Convergence of Policy Gradient Methods
Program in Applied and Computational Mathematics, Princeton（普林斯顿大学应用与计算数学项目）；Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning（上海人工智能与深度学习前沿科学中心）；NYU-ECNU Institute of Mathematical Sciences at NYU Shanghai（纽约大学-华东师范大学纽约大学上海数学科学研究所）；NYU Shanghai（上海纽约大学）；Walmart Global Tech（沃尔玛全球科技）
Interdisciplinary Graduate School, Nanyang Technological University Singapore（南洋理工大学跨学科研究生院（新加坡））；School of Electrical and Electronic Engineering, Nanyang Technological University Singapore（南洋理工大学电气与电子工程学院（新加坡））
Mobile energy storage systems (MESSs) provide mobility and flexibility to enhance distribution system resilience. The paper proposes a Markov decision process (MDP) formulation for an integrated service restoration strategy that coordinates the scheduling of MESSs and resource dispatching of microgrids. The uncertainties in load consumption are taken into account. The deep reinforcement learning (DRL) algorithm is utilized to solve the MDP for optimal scheduling. Specifically, the twin delayed deep deterministic policy gradient (TD3) is applied to train the deep Q-network and policy network, t…
A Sliding-Window-Based Reinforcement Learning for Dynamic Assembly Flow Shop Scheduling with Multi-Product Delivery
School of Electromechanical Engineering, Guangdong University of Technology（广东工业大学机电工程学院）；Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）
Constrained Reinforcement Learning for Dynamic Material Handling
Research Institute of Trustworthy Autonomous Systems (RITAS), Southern University of Science and Technology（南方科技大学可信自主系统研究院）；Southern University of Science and Technology（南方科技大学）；Trustworthiness Theory Research Center, Huawei Technologies Co., Ltd（华为技术有限公司可信理论研究中心）
