arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2008.00030eess.SYcs.LGcs.SY
收藏

过程控制与优化中的机会约束策略优化

Chance Constrained Policy Optimization for Process Control and Optimization

University College London(伦敦大学学院);Imperial College London(伦敦帝国学院);Norwegian University of Science and Technology(挪威科技大学);The University of Manchester(曼彻斯特大学)

Panagiotis Petsagkourakis, Ilya Orson Sandoval, Eric Bradford, Federico Galvanin, Dongda Zhang, Ehecatl Antonio del Rio-Chanona

摘要:

化工过程的优化和控制受到以下因素的影响:1)工厂模型失配,2)过程扰动,3)安全操作的约束。通过策略优化的强化学习将是解决这个问题的自然方法,因为它能够解决随机性,工厂模型失配,并以适当的闭环方式直接考虑未来不确定性及其反馈的影响;所有这些都不需要内部优化循环。强化学习尚未被考虑用于工业过程(或几乎任何工程应用)的主要原因之一是它缺乏处理安全关键约束的框架。目前的策略优化算法使用难以调整的惩罚参数,不能可靠地满足状态约束或目前的保证,只有在预期。我们提出了一个机会约束策略优化(CCPO)算法,保证满足联合机会约束的高概率-这是至关重要的安全关键任务。这是通过引入约束收紧(退避)来实现的,约束收紧与反馈策略同时计算。回退使用概率约束的经验累积分布函数通过贝叶斯优化进行调整,因此是自调整的。这导致在一个一般的方法,可以注入到目前的政策优化算法,使他们能够满足联合机会约束的高概率。我们提出的案例研究,分析所提出的方法的性能。

英文摘要:

Chemical process optimization and control are affected by 1) plant-model mismatch, 2) process disturbances, and 3) constraints for safe operation. Reinforcement learning by policy optimization would be a natural way to solve this due to its ability to address stochasticity, plant-model mismatch, and directly account for the effect of future uncertainty and its feedback in a proper closed-loop manner; all without the need of an inner optimization loop. One of the main reasons why reinforcement learning has not been considered for industrial processes (or almost any engineering application) is that it lacks a framework to deal with safety critical constraints. Present algorithms for policy optimization use difficult-to-tune penalty parameters, fail to reliably satisfy state constraints or present guarantees only in expectation. We propose a chance constrained policy optimization (CCPO) algorithm which guarantees the satisfaction of joint chance constraints with a high probability - which is crucial for safety critical tasks. This is achieved by the introduction of constraint tightening (backoffs), which are computed simultaneously with the feedback policy. Backoffs are adjusted with Bayesian optimization using the empirical cumulative distribution function of the probabilistic constraints, and are therefore self-tuned. This results in a general methodology that can be imbued into present policy optimization algorithms to enable them to satisfy joint chance constraints with high probability. We present case studies that analyze the performance of the proposed approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑