arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.02205cs.LG
收藏

使用基于模型的强化学习进行数字孪生校准

Digital Twin Calibration with Model-Based Reinforcement Learning

Northeastern University(东北大学);University of Maryland(马里兰大学)

Hua Zheng, Wei Xie, Ilya O. Ryzhov, Keilung Choy

AI总结:

提出 Actor-Simulator 框架,将数字孪生校准与基于模型的强化学习相结合,联合优化孪生模型与控制策略,并在生物制药制造实验中验证其收敛性和优越性能。

摘要:

本文提出了一个名为 Actor-Simulator 的新型方法框架,该框架将数字孪生校准纳入基于模型的强化学习中,以便更有效地控制具有复杂非线性动力学的随机系统。传统的基于模型的控制通常依赖于限制性的结构假设(例如线性状态转移),并且未能考虑模型中的参数不确定性。这些问题在生物制药制造等行业中变得尤为关键,在这些行业中,过程动力学复杂且尚未完全已知,并且仅有有限数量的数据可用。我们的方法联合校准数字孪生并搜索最优控制策略,从而考虑并减少模型误差。我们通过使用策略性能作为数据收集的指导来平衡探索与利用。这种双组件方法可证明收敛到最优策略,并且在基于生物制药制造领域的大量数值实验中优于现有方法。

英文摘要:

This paper presents a novel methodological framework, called the Actor-Simulator, that incorporates the calibration of digital twins into model-based reinforcement learning for more effective control of stochastic systems with complex nonlinear dynamics. Traditional model-based control often relies on restrictive structural assumptions (such as linear state transitions) and fails to account for parameter uncertainty in the model. These issues become particularly critical in industries such as biopharmaceutical manufacturing, where process dynamics are complex and not fully known, and only a limited amount of data is available. Our approach jointly calibrates the digital twin and searches for an optimal control policy, thus accounting for and reducing model error. We balance exploration and exploitation by using policy performance as a guide for data collection. This dual-component approach provably converges to the optimal policy, and outperforms existing methods in extensive numerical experiments based on the biopharmaceutical manufacturing domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑