arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.15091cs.LGcs.AIcs.SYeess.SY
收藏

用于深度强化学习的改进型长短期记忆网络污水处理模拟器

Improved Long Short-Term Memory-based Wastewater Treatment Simulators for Deep Reinforcement Learning

Krüger A/S(克鲁格公司);Aalborg University(奥尔堡大学);AAU Energy(奥尔堡大学能源学院)

Esmaeel Mohammadi, Daniel Ortiz-Arroyo, Mikkel Stokholm-Bjerregaard, Aviaja Anna Hansen, Petar Durdevic

AI总结:

针对污水处理数据非线性导致DRL模拟器存在复合误差的问题,提出在训练中引入模型预测作为输入校正并改进损失函数以考虑长期动态,使模拟器全年预测准确度最高提升98%。

摘要:

尽管深度强化学习(DRL)在机器人和游戏领域展现出卓越成果,但将其应用于污水处理等工业过程的优化仍具挑战。挑战之一在于缺乏能尽可能准确表示实际工厂以训练DRL策略的模拟环境。污水处理数据的随机性与非线性导致模型在长时间范围内的预测不稳定且不正确。模型模拟行为不正确的一个可能原因与复合误差问题有关,即误差在整个模拟过程中的累积。复合误差的发生是因为模型在每个时间步将其预测用作输入。随着模拟的进行,实际数据与预测之间的误差不断累积。我们实施了两种方法来改进针对污水处理数据训练的模型,从而获得了更准确的模拟器:1-在训练步骤中将模型的预测数据作为输入以作为校正工具;2-改变损失函数以考虑长期预测形状(动态特性)。实验结果表明,与基础模型相比,实施这些方法可在全年的动态时间规整(Dynamic Time Warping)指标上将模拟器行为改善高达98%。这些改进在为生物过程创建模拟器方面展现出重大前景,此类模拟器无需预先了解该过程,而是完全依赖于从系统获取的时间序列数据。

英文摘要:

Even though Deep Reinforcement Learning (DRL) showed outstanding results in the fields of Robotics and Games, it is still challenging to implement it in the optimization of industrial processes like wastewater treatment. One of the challenges is the lack of a simulation environment that will represent the actual plant as accurately as possible to train DRL policies. Stochasticity and non-linearity of wastewater treatment data lead to unstable and incorrect predictions of models over long time horizons. One possible reason for the models' incorrect simulation behavior can be related to the issue of compounding error, which is the accumulation of errors throughout the simulation. The compounding error occurs because the model utilizes its predictions as inputs at each time step. The error between the actual data and the prediction accumulates as the simulation continues. We implemented two methods to improve the trained models for wastewater treatment data, which resulted in more accurate simulators: 1- Using the model's prediction data as input in the training step as a tool of correction, and 2- Change in the loss function to consider the long-term predicted shape (dynamics). The experimental results showed that implementing these methods can improve the behavior of simulators in terms of Dynamic Time Warping throughout a year up to 98% compared to the base model. These improvements demonstrate significant promise in creating simulators for biological processes that do not need pre-existing knowledge of the process but instead depend exclusively on time series data obtained from the system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑