arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.14972math.OC
收藏

部分可观测马尔可夫决策过程的学习与控制分离

On Separation Between Learning and Control in Partially Observed Markov Decision Processes

Andreas A. Malikopoulos

摘要:

信息物理系统(CPS)遇到大量的数据,这些数据是实时地逐渐添加到系统中的,而不是完全提前。随着数据量的增加,控制策略的域也增加,因此搜索最优策略变得具有挑战性。即使找到了最优控制策略,在增加域的情况下实施这样的策略也是繁重的。为了推导CPS中的最优控制策略,我们通常假设系统的理想模型。由于模型与实际CPS之间的差异,这种基于模型的控制方法不能有效地促进具有性能保证的最优解。或者,传统的监督学习方法不能总是使用离线导出的数据来促进鲁棒的解决方案。同样,将强化学习方法直接应用于实际CPS可能会对系统的安全性和鲁棒性产生重大影响。本章的目标是提供一个理论框架,旨在分离的控制和学习任务,使我们能够结合离线基于模型的控制与在线学习的方法,从而规避挑战,在推导CPS的最优控制策略。

英文摘要:

Cyber-physical systems (CPS) encounter a large volume of data which is added to the system gradually in real time and not altogether in advance. As the volume of data increases, the domain of the control strategies also increases, and thus it becomes challenging to search for an optimal strategy. Even if an optimal control strategy is found, implementing such strategies with increasing domains is burdensome. To derive an optimal control strategy in CPS, we typically assume an ideal model of the system. Such model-based control approaches cannot effectively facilitate optimal solutions with performance guarantees due to the discrepancy between the model and the actual CPS. Alternatively, traditional supervised learning approaches cannot always facilitate robust solutions using data derived offline. Similarly, applying reinforcement learning approaches directly to the actual CPS might impose significant implications on safety and robust operation of the system. The goal of this chapter is to provide a theoretical framework that aims at separating the control and learning tasks which allows us to combine offline model-based control with online learning approaches, and thus circumvent the challenges in deriving optimal control strategies for CPS.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑