自动调度是制造业中广泛需要的一种调度方法，其目标是在有限的资源范围内优化作业顺序。然而，现有的调度系统严重依赖于启发式算法，这要么产生无效的解决方案或计算效率低下时，作业规模的增加。因此，开发新的大规模算法，不仅是高效和有效的，而且能够满足复杂的约束在实践中是非常重要的。在本文中，我们提出了一个双层深度强化学习模型，\textit{BDS}，其中较高的水平是负责探索一个初始的全局序列，而较低的水平是旨在开发部分序列的改进，这两个层次是通过滑动窗口采样机制连接。在实现中，双深度Q网络（DDQN）用于上层，图指针网络（GPN）位于下层。在理论上保证了BDS的收敛性之后，我们在工业自动化仓库的场景中对其进行了评估，每条生产线上的作业数高达$5000$。结果表明，我们提出的BDS显著优于两种最常用的算法，三种强大的深度网络和另一种双层基线方法。特别是，与现实世界中最常用的基于贪婪的启发式算法（需要近1个小时）相比，我们的BDS在3个最大数据集上的最大完工时间分别减少了27.5%、28.6%和22.1%，计算时间小于200 s.
英文摘要：
Automatic industrial scheduling, aiming at optimizing the sequence of jobs over limited resources, is widely needed in manufacturing industries. However, existing scheduling systems heavily rely on heuristic algorithms, which either generate ineffective solutions or compute inefficiently when job scale increases. Thus, it is of great importance to develop new large-scale algorithms that are not only efficient and effective, but also capable of satisfying complex constraints in practice. In this paper, we propose a Bilevel Deep reinforcement learning Scheduler, \textit{BDS}, in which the higher level is responsible for exploring an initial global sequence, whereas the lower level is aiming at exploitation for partial sequence refinements, and the two levels are connected by a sliding-window sampling mechanism. In the implementation, a Double Deep Q Network (DDQN) is used in the upper level and Graph Pointer Network (GPN) lies within the lower level. After the theoretical guarantee for the convergence of BDS, we evaluate it in an industrial automatic warehouse scenario, with job number up to $5000$ in each production line. It is shown that our proposed BDS significantly outperforms two most used heuristics, three strong deep networks, and another bilevel baseline approach. In particular, compared with the most used greedy-based heuristic algorithm in real world which takes nearly an hour, our BDS can decrease the makespan by 27.5\%, 28.6\% and 22.1\% for 3 largest datasets respectively, with computational time less than 200 seconds.
An Intelligent Deterministic Scheduling Method for Ultra-Low Latency Communication in Edge Enabled Industrial Internet of Things
Chongqing University of Posts and Telecommunications（重庆邮电大学）；University of Guelph（圭尔夫大学）；Hubei University of Arts and Science（湖北文理学院）；Hosei University（法政大学）；RIKEN（理化学研究所）
