尽管深度强化学习(DRL)在机器人和游戏领域展现出卓越成果,但将其应用于污水处理等工业过程的优化仍具挑战。挑战之一在于缺乏能尽可能准确表示实际工厂以训练DRL策略的模拟环境。污水处理数据的随机性与非线性导致模型在长时间范围内的预测不稳定且不正确。模型模拟行为不正确的一个可能原因与复合误差问题有关,即误差在整个模拟过程中的累积。复合误差的发生是因为模型在每个时间步将其预测用作输入。随着模拟的进行,实际数据与预测之间的误差不断累积。我们实施了两种方法来改进针对污水处理数据训练的模型,从而获得了更准确的模拟器:1-在训练步骤中将模型的预测数据作为输入以作为校正工具;2-改变损失函数以考虑长期预测形状(动态特性)。实验结果表明,与基础模型相比,实施这些方法可在全年的动态时间规整(Dynamic Time Warping)指标上将模拟器行为改善高达98%。这些改进在为生物过程创建模拟器方面展现出重大前景,此类模拟器无需预先了解该过程,而是完全依赖于从系统获取的时间序列数据。
英文摘要:
Even though Deep Reinforcement Learning (DRL) showed outstanding results in the fields of Robotics and Games, it is still challenging to implement it in the optimization of industrial processes like wastewater treatment. One of the challenges is the lack of a simulation environment that will represent the actual plant as accurately as possible to train DRL policies. Stochasticity and non-linearity of wastewater treatment data lead to unstable and incorrect predictions of models over long time horizons. One possible reason for the models' incorrect simulation behavior can be related to the issue of compounding error, which is the accumulation of errors throughout the simulation. The compounding error occurs because the model utilizes its predictions as inputs at each time step. The error between the actual data and the prediction accumulates as the simulation continues. We implemented two methods to improve the trained models for wastewater treatment data, which resulted in more accurate simulators: 1- Using the model's prediction data as input in the training step as a tool of correction, and 2- Change in the loss function to consider the long-term predicted shape (dynamics). The experimental results showed that implementing these methods can improve the behavior of simulators in terms of Dynamic Time Warping throughout a year up to 98% compared to the base model. These improvements demonstrate significant promise in creating simulators for biological processes that do not need pre-existing knowledge of the process but instead depend exclusively on time series data obtained from the system.
A Hybrid Approach of Transfer Learning and Physics-Informed Modeling: Improving Dissolved Oxygen Concentration Prediction in an Industrial Wastewater Treatment Plant
Explainable Uncertainty Quantification for Wastewater Treatment Energy Prediction via Interval Type-2 Neuro-Fuzzy System
Jheronimus Academy of Data Science, Eindhoven University of Technology, Eindhoven, The Netherlands(杰罗尼穆斯数据科学学院,埃因霍温理工大学,埃因霍温,荷兰);Haskoning, Amersfoort, The Netherlands(哈索宁,阿默斯福尔特,荷兰);School of Industrial Engineering, Eindhoven University of Technology(工业工程学院,埃因霍温理工大学)
Medical world models: representing medical states, modelling clinical dynamics and guiding intervention policies
College of Computer Science, Zhejiang University(浙江大学计算机科学与技术学院);School of Medicine, Zhejiang University(浙江大学医学院);Department of Biomedical Informatics, Harvard University(哈佛大学生物医学信息学系)
Physics-informed reservoir characterization from bulk and extreme pressure events with a differentiable simulator
Earth and Environmental Sciences Division, Los Alamos National Laboratory(洛斯阿拉莫斯国家实验室地球与环境科学部);Courant Institute School of Mathematics, Computing and Data Science, New York University(纽约大学柯朗研究所数学、计算与数据科学学院)
Organ-Agents: Virtual Human Physiology Simulator via LLMs
Tianjin University(天津大学);University of Trento(特伦托大学);Tianjin Medical University General Hospital(天津医科大学总医院);Chest Hospital, Tianjin University(天津大学胸科医院);The Affiliated Suzhou Hospital of Nanjing Medical University(南京医科大学附属苏州医院);Fondazione Bruno Kessler(布鲁诺·凯斯勒基金会)