arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新，休息一下吧，周末愉快~~
arXiv 2601.09293cs.AI
收藏

基于行动掩码的策略优化强化学习在不确定动态作业车间调度中的应用：处理随机到达和机器故障

Policy-Based Reinforcement Learning with Action Masking for Dynamic Job Shop Scheduling under Uncertainty: Handling Random Arrivals and Machine Failures

South Westphalia University of Applied Sciences（西南弗劳恩霍夫应用科学大学）

Sofiene Lassoued, Stefan Lier, Andreas Schwung

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出了一种结合可解释Petri网模型与自适应强化学习策略的动态作业车间调度方法，通过行动掩码策略有效应对随机到达和机器故障问题，提升调度的鲁棒性和可解释性。

摘要：

我们提出了一种新的框架，用于解决在不确定条件下动态作业车间调度问题，以应对随机作业到达和突发机器故障带来的挑战。我们的方法采用基于模型的范式，利用彩色时标Petri网来表示调度环境，并利用可掩码的近端策略优化来实现动态决策，同时在每个决策点限制智能体只能执行可行的动作。为了模拟真实的工业条件，动态作业到达被建模为伽马分布，该分布能够捕捉复杂的时序模式，如突发、聚类和波动的工作负载。机器故障被建模为韦布尔分布，以表示年龄相关的退化和磨损动态。这些随机模型使框架能够更好地反映现实中的制造场景。此外，我们研究了两种行动掩码策略：一种非梯度方法，它会覆盖无效动作的概率，另一种基于梯度的方法，它在策略网络内为无效动作分配负梯度。我们在动态JSSP基准上进行了广泛的实验，证明我们的方法在最小化完成时间方面始终优于传统的启发式和规则方法。结果突显了将可解释的Petri网模型与自适应强化学习策略相结合的优势，从而获得一个稳健、可扩展且可解释的框架，用于动态和不确定制造环境中的实时调度。

英文摘要：

We present a novel framework for solving Dynamic Job Shop Scheduling Problems under uncertainty, addressing the challenges introduced by stochastic job arrivals and unexpected machine breakdowns. Our approach follows a model-based paradigm, using Coloured Timed Petri Nets to represent the scheduling environment, and Maskable Proximal Policy Optimization to enable dynamic decision-making while restricting the agent to feasible actions at each decision point. To simulate realistic industrial conditions, dynamic job arrivals are modeled using a Gamma distribution, which captures complex temporal patterns such as bursts, clustering, and fluctuating workloads. Machine failures are modeled using a Weibull distribution to represent age-dependent degradation and wear-out dynamics. These stochastic models enable the framework to reflect real-world manufacturing scenarios better. In addition, we study two action-masking strategies: a non-gradient approach that overrides the probabilities of invalid actions, and a gradient-based approach that assigns negative gradients to invalid actions within the policy network. We conduct extensive experiments on dynamic JSSP benchmarks, demonstrating that our method consistently outperforms traditional heuristic and rule-based approaches in terms of makespan minimization. The results highlight the strength of combining interpretable Petri-net-based models with adaptive reinforcement learning policies, yielding a resilient, scalable, and explainable framework for real-time scheduling in dynamic and uncertain manufacturing environments.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑