基于线性传递算子的数据驱动随机最优控制
Data-Driven Stochastic Optimal Control using Linear Transfer Operators
摘要：
我们提供了一个数据驱动的框架，连续时间随机动力系统的最优控制。所提出的框架依赖于涉及线性Perron-Frobenius（P-F）和Koopman算子的线性算子理论。我们的第一个结果涉及P-F算子提供了一个凸的制定最优控制问题的对偶空间的密度。这种随机最优控制问题的凸形式导致一个无穷维凸规划。使用P-F算子的数据驱动近似获得凸规划的有限维近似。我们的第二个结果表明，使用Koopman算子，这是对偶的P-F算子，随机最优控制设计。我们证明了Hamilton Jacobi Bellman（HJB）方程可以用Koopman算子表示。我们提供了一个迭代过程，沿着一个流行的政策迭代算法的基础上的数据驱动近似的Koopman算子求解HJB方程。这两个公式，即涉及P-F算子的凸公式和使用HJB方程的Koopman公式，可以被视为彼此对偶，其中由于P-F和Koopman算子的对偶性质，对偶性随之而来。最后，我们提出了几个数值例子来证明所开发的框架的有效性。
英文摘要：
We provide a data-driven framework for optimal control of a continuous-time stochastic dynamical system. The proposed framework relies on the linear operator theory involving linear Perron-Frobenius (P-F) and Koopman operators. Our first results involving the P-F operator provide a convex formulation to the optimal control problem in the dual space of densities. This convex formulation of the stochastic optimal control problem leads to an infinite-dimensional convex program. The finite-dimensional approximation of the convex program is obtained using a data-driven approximation of the P-F operator. Our second results demonstrate the use of the Koopman operator, which is dual to the P-F operator, for the stochastic optimal control design. We show that the Hamilton Jacobi Bellman (HJB) equation can be expressed using the Koopman operator. We provide an iterative procedure along the lines of a popular policy iteration algorithm based on the data-driven approximation of the Koopman operator for solving the HJB equation. The two formulations, namely the convex formulation involving P-F operator and Koopman based formulation using HJB equation, can be viewed as dual to each other where the duality follows due to the dual nature of P-F and Koopman operators. Finally, we present several numerical examples to demonstrate the efficacy of the developed framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
