When designing optimal controllers for any system, it is often the case that the true state of the system is unknown to the controller, for example due to noisy measurements or partially observable states. Incomplete state information must be taken into account in the controller's design in order to preserve its optimality. The same is true when performing reachability calculations. To estimate the probability that the state of a stochastic system reaches, or stays within, some set of interest in a given time horizon, it is necessary to find a controller (or at least prove one exists) that drives the system to that set with maximum probability. This controller, however, does not have access to the true state of the system. To date, little work has been done on stochastic reachability calculations with partially observable states. What work has been done relies on converting the reachability optimization problem to one with an additive cost function, for which theoretical results are well known. Our approach is to preserve the multiplicative cost structure when deriving a sufficient statistic that reduces the problem to one of perfect state information. Our transformation includes a change of measure that simplifies the distribution of the sufficient statistic conditioned on its previous value. We develop a dynamic programming recursion for the solution of the equivalent perfect information problem, proving that the recursion is valid, an optimal solution exists, and results in the same solution as to the original problem. We also show that our results are equivalent to those for the reformulated additive cost problem, and so such a reformulation is not required.
Pontryagin's Minimum Principle and Forward-Backward Sweep Method for the System of HJB-FP Equations in Memory-Limited Partially Observable Stochastic Control
Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes
Max Planck Institute for Mathematics in the Sciences(马克斯·普朗克数学研究所);Faculty of Mathematics and Computer Science(数学与计算机科学学院);Leipzig University(莱比锡大学);Santa Fe Institute(圣塔菲研究所)