具有无限策略的压缩模型的随机化λ策略迭代:适定性和收敛性(扩展版)
Lambda-Policy Iteration with Randomization for Contractive Models with Infinite Policies: Well-Posedness and Convergence (Extended Version)
摘要:
抽象动态规划模型用于分析$λ$ -策略迭代与随机算法。特别是,收缩模型与无限的政策被认为是它示出的适定性的$λ$ -运营商在算法中起着核心作用。该算子对于有限状态的问题是适定的,但我们的分析表明,对于无限状态的压缩模型,该算子也是适定的。同样,我们分析的算法已知对于有限策略的问题会收敛,但我们确定了当策略空间无限时(无论状态数量如何)保证以概率一收敛所需的条件。在分析的指导下,我们提出了一个数据驱动的近似实现算法的约束线性和非线性控制问题的最优成本估计。数值结果表明了该方法在实际应用中的潜力。
英文摘要:
Abstract dynamic programming models are used to analyze $λ$-policy iteration with randomization algorithms. Particularly, contractive models with infinite policies are considered and it is shown that well-posedness of the $λ$-operator plays a central role in the algorithm. The operator is known to be well-posed for problems with finite states, but our analysis shows that it is also well-defined for the contractive models with infinite states studied. Similarly, the algorithm we analyze is known to converge for problems with finite policies, but we identify the conditions required to guarantee convergence with probability one when the policy space is infinite regardless of the number of states. Guided by the analysis, we exemplify a data-driven approximated implementation of the algorithm for estimation of optimal costs of constrained linear and nonlinear control problems. Numerical results indicate potentials of this method in practice.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。