Optimal control deals with optimization problems in which variables steer a dynamical system, and its outcome contributes to the objective function. Two classical approaches to solving these problems are Dynamic Programming and the Pontryagin Maximum Principle. In both approaches, Hamiltonian equations offer an interpretation of optimality through auxiliary variables known as costates. However, Hamiltonian equations are rarely used due to their reliance on forward-backward algorithms across the entire temporal domain. This paper introduces a novel neural-based approach to optimal control, with the aim of working forward-in-time. Neural networks are employed not only for implementing state dynamics but also for estimating costate variables. The parameters of the latter network are determined at each time step using a newly introduced local policy referred to as the time-reversed generalized Riccati equation. This policy is inspired by a result discussed in the Linear Quadratic (LQ) problem, which we conjecture stabilizes state dynamics. We support this conjecture by discussing experimental results from a range of optimal control case studies.
The Discrete-Time Generalized Algebraic Riccati Equation: Order Reduction and Solutions' Structure
本文讨论了如何将最优控制和最优滤波问题中的约束广义离散时间代数Riccati方程分解为两部分,对应于每个解X的加法分解X= X 0 +D:第一部分是加数X 0的显式表达式,它是所有解的公共加数,不依赖于特定的X。第二部分可以是一个降阶离散时间正则代数Riccati方程,其相关的闭环矩阵是非奇异的,或对称Stein方程。
A Convergent Algorithm Based on Deterministic Approximation for a Large Class of Regime-Switching Generalized Stochastic Game-Theoretic Riccati Differential Equations
Shandong University Zhongtai Securities Institute for Financial Studies, Shandong University(山东大学中泰证券金融研究院,山东大学)