arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.18018math.OC
收藏

离散时间部分观测下的连续时间随机最优控制

Continuous time Stochastic optimal control under discrete time partial observations

Christian Bayer, Boualem Djehiche, Eliza Rezvanova, Raul Fidel Tempone

AI总结:

本文针对连续时间状态演化、离散时间部分带噪观测的随机最优控制问题,提出基于概率测度值状态和贝叶斯更新的控制框架,并通过交错HJB方程及高斯情形下的有限维等价方程刻画最优策略,数值实验验证了其有效性。

摘要:

本文研究一类随机最优控制问题,其中未知状态在连续时间内演化,而部分、带噪且可能可控的测量仅在离散时间可用。我们建立了一个用于控制此类系统的框架,重点关注系统状态的概率测度值过程以及依赖于带噪和不完整数据的控制动作。我们的方法采用以概率测度值为状态的随机最优控制框架,该框架能够容纳带噪测量,并通过贝叶斯更新机制将测量信息整合到控制决策中。我们将控制最优性刻画为一组交错排列的 Hamilton Jacobi Bellman(HJB)方程序列,并在测量时刻耦合受控脉冲步骤。针对高斯受控过程的情形,我们推导出一个等价的 HJB 方程,其状态变量为有限维的,即状态的均值和协方差。我们通过数值算例验证了所提方法的有效性,这些算例包括完全观测下的控制、无观测下的控制以及带噪观测下的控制。数值结果揭示了不同控制策略及其性能之间的显著差异,突显了处理状态观测不确定性所面临的挑战和计算需求。

英文摘要:

This work addresses stochastic optimal control problems where the unknown state evolves in continuous time while partial, noisy, and possibly controllable measurements are only available in discrete time. We develop a framework for controlling such systems, focusing on the measure-valued process of the system's state and the control actions that depend on noisy and incomplete data. Our approach uses a stochastic optimal control framework with a probability measure-valued state, which accommodates noisy measurements and integrates them into control decisions through a Bayesian update mechanism. We characterize the control optimality in terms of a sequence of interlaced Hamilton Jacobi Bellman (HJB) equations coupled with controlled impulse steps at the measurement times. For the case of Gaussian-controlled processes, we derive an equivalent HJB equation whose state variable is finite-dimensional, namely the state's mean and covariance. We demonstrate the effectiveness of our methods through numerical examples. These include control under perfect observations, control under no observations, and control under noisy observations. Our numerical results highlight significant differences in the control strategies and their performance, emphasizing the challenges and computational demands of dealing with uncertainty in state observation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑