一种使用在线与离线深度强化学习的维护规划框架
A Maintenance Planning Framework using Online and Offline Deep Reinforcement Learning
AI总结：
本文提出一种基于在线与离线深度强化学习的水管修复维护规划框架，利用DQN和保守Q学习从模拟交互及静态回放数据中学习最优策略，实验证明其优于传统预防性、纠正性和贪心规划方法。
摘要：
具有成本效益的资产管理是多个行业共同关注的领域。具体而言，本文提出了一种深度强化学习（DRL）解决方案，用于自动确定持续劣化水管的优化修复策略。我们从在线和离线DRL两种设定来研究修复规划问题。在在线DRL中，智能体与一个包含多根具有不同长度、材料和故障率特性的水管的模拟环境进行交互。我们使用深度Q学习（DQN）训练智能体，以学习在平均成本最低且故障概率降低情况下的最优策略。在离线学习中，智能体利用静态数据（例如DQN回放数据），通过保守Q学习算法学习最优策略，而无需与环境进行进一步交互。我们证明，基于DRL的策略优于标准的预防性、纠正性和贪心规划替代方案。此外，在离线设定中从固定的DQN回放数据集进行学习进一步提升了性能。结果表明，水管现有的劣化剖面包含大量多样的状态和动作轨迹，为在离线设定中学习修复策略提供了有价值的途径，并且这些策略可以进一步利用模拟器进行微调。
英文摘要：
Cost-effective asset management is an area of interest across several industries. Specifically, this paper develops a deep reinforcement learning (DRL) solution to automatically determine an optimal rehabilitation policy for continuously deteriorating water pipes. We approach the problem of rehabilitation planning in an online and offline DRL setting. In online DRL, the agent interacts with a simulated environment of multiple pipes with distinct lengths, materials, and failure rate characteristics. We train the agent using deep Q-learning (DQN) to learn an optimal policy with minimal average costs and reduced failure probability. In offline learning, the agent uses static data, e.g., DQN replay data, to learn an optimal policy via a conservative Q-learning algorithm without further interactions with the environment. We demonstrate that DRL-based policies improve over standard preventive, corrective, and greedy planning alternatives. Additionally, learning from the fixed DQN replay dataset in an offline setting further improves the performance. The results warrant that the existing deterioration profiles of water pipes consisting of large and diverse states and action trajectories provide a valuable avenue to learn rehabilitation policies in the offline setting, which can be further fine-tuned using the simulator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
