强化学习为现实的供应链安全库存优化提供了灵活的方法
Reinforcement Learning Provides a Flexible Approach for Realistic Supply Chain Safety Stock Optimisation
摘要:
虽然安全库存优化已经研究了60多年,但大多数公司仍然使用简单的方法来计算必要的安全库存水平,部分原因是现有分析方法的重点是导出可证明的最佳解决方案,而公司偏好牺牲最佳结果以支持更现实的问题设置。人工智能(AI)领域的一种新兴方法,即强化学习(RL),提供了在找到最优解的同时适应更现实的问题特征的希望。与基于分析的模型不同,RL将问题视为黑盒仿真环境,以缓解过度简化现实的问题。因此,库存政策的假设可以放宽,可以容纳更多的问题变量。虽然RL在其他领域很受欢迎,但其在安全库存优化中的应用仍然很少。在本文中,我们研究了三种RL方法,即Q学习,时间差优势演员-评论家和多代理时间差优势演员-评论家,用于优化独立代理线性链中的安全库存。我们发现,RL可以同时优化安全库存水平和订货量参数的库存政策,不像经典的安全库存优化模型,只有安全库存水平进行优化,而订货量是预先确定的基础上简单的规则。这使得RL能够对更复杂的供应链采购行为进行建模。然而,强化学习需要更长的时间来获得解决方案,因此需要进一步研究如何识别和改进人工智能和数学模型之间的权衡。
英文摘要:
Although safety stock optimisation has been studied for more than 60 years, most companies still use simplistic means to calculate necessary safety stock levels, partly due to the mismatch between existing analytical methods' emphases on deriving provably optimal solutions and companies' preferences to sacrifice optimal results in favour of more realistic problem settings. A newly emerging method from the field of Artificial Intelligence (AI), namely Reinforcement Learning (RL), offers promise in finding optimal solutions while accommodating more realistic problem features. Unlike analytical-based models, RL treats the problem as a black-box simulation environment mitigating against the problem of oversimplifying reality. As such, assumptions on stock keeping policy can be relaxed and a higher number of problem variables can be accommodated. While RL has been popular in other domains, its applications in safety stock optimisation remain scarce. In this paper, we investigate three RL methods, namely, Q-Learning, Temporal Difference Advantage Actor-Critic and Multi-agent Temporal Difference Advantage Actor-Critic for optimising safety stock in a linear chain of independent agents. We find that RL can simultaneously optimise both safety stock level and order quantity parameters of an inventory policy, unlike classical safety stock optimisation models where only safety stock level is optimised while order quantity is predetermined based on simple rules. This allows RL to model more complex supply chain procurement behaviour. However, RL takes longer time to arrive at solutions, necessitating future research on identifying and improving trade-offs between the use of AI and mathematical models are needed.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。