arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.09598math.OCcs.SYeess.SY
收藏

动态输出反馈线性二次控制的优化景观研究

On the Optimization Landscape of Dynamic Output Feedback Linear Quadratic Control

Jingliang Duan, Wenhan Cao, Yang Zheng, Lin Zhao

AI总结:

本文研究了动态输出反馈线性二次调节(dLQR)的优化景观,证明了可观测dLQR平稳点的唯一性,并建立了其与LQG控制的等价条件,为策略梯度算法求解动态控制器提供了理论保证。

摘要:

策略梯度算法的收敛性取决于底层最优控制问题的优化景观。对这些算法的理论认识通常可以通过分析线性二次控制问题来获得。然而,现有文献大多只考虑静态全状态或输出反馈策略(控制器)的优化景观。我们研究了更具挑战性的动态输出反馈策略下的线性二次调节问题(简称dLQR),该问题在实践中普遍存在,但其优化景观相当复杂。我们首先展示了dLQR代价如何随动态控制器的坐标变换而变化,然后针对给定的可观测镇定控制器推导出最优变换。我们的核心结果之一是:当dLQR可观测时,其平稳点是唯一的,这为使用策略梯度方法求解动态控制器提供了最优性证明。此外,我们建立了dLQR与线性二次高斯控制等价的条件,从而为确定性和随机线性系统的最优控制提供了统一的视角。这些结果进一步为设计面向部分观测信息的更一般决策问题的策略梯度算法提供了启示。

英文摘要:

The convergence of policy gradient algorithms hinges on the optimization landscape of the underlying optimal control problem. Theoretical insights into these algorithms can often be acquired from analyzing those of linear quadratic control. However, most of the existing literature only considers the optimization landscape for static full-state or output feedback policies (controllers). We investigate the more challenging case of dynamic output-feedback policies for linear quadratic regulation (abbreviated as dLQR), which is prevalent in practice but has a rather complicated optimization landscape. We first show how the dLQR cost varies with the coordinate transformation of the dynamic controller and then derive the optimal transformation for a given observable stabilizing controller. One of our core results is the uniqueness of the stationary point of dLQR when it is observable, which provides an optimality certificate for solving dynamic controllers using policy gradient methods. Moreover, we establish conditions under which dLQR and linear quadratic Gaussian control are equivalent, thus providing a unified viewpoint of optimal control of both deterministic and stochastic linear systems. These results further shed light on designing policy gradient algorithms for more general decision-making problems with partially observed information.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑