arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.05596eess.SYcs.SYmath.OC
收藏

面向线性二次跟踪的数据驱动策略优化的线性收敛性

Linear Convergence of Data-Enabled Policy Optimization for Linear Quadratic Tracking

Shubo Kang, Feiran Zhao, Keyou You

AI总结:

本研究将数据驱动策略优化(DeePO)框架拓展至带离线数据的线性二次跟踪问题,通过策略协方差参数化得到数据驱动表述,用梯度下降迭代求解最优策略,证明了该迭代的线性收敛性并经数值实验验证,为直接自适应LQT研究奠定基础。

摘要:

数据驱动策略优化(DeePO)是新近提出的用于解决直接自适应LQR这一开放问题的方法。本文将DeePO框架拓展至带离线数据的线性二次跟踪(LQT)问题。通过引入LQT策略的协方差参数化,我们推导得到LQT问题的直接数据驱动表述。随后,我们采用梯度下降法迭代更新参数化策略,以求解最优LQT策略。此外,通过揭示DeePO与基于模型的策略优化之间的关联,我们证明了DeePO迭代的线性收敛性。最后,我们通过数值实验验证了收敛结果。我们希望本研究为利用在线闭环数据的直接自适应LQT奠定基础。

英文摘要:

Data-enabled policy optimization (DeePO) is a newly proposed method to attack the open problem of direct adaptive LQR. In this work, we extend the DeePO framework to the linear quadratic tracking (LQT) with offline data. By introducing a covariance parameterization of the LQT policy, we derive a direct data-driven formulation of the LQT problem. Then, we use gradient descent method to iteratively update the parameterized policy to find an optimal LQT policy. Moreover, by revealing the connection between DeePO and model-based policy optimization, we prove the linear convergence of the DeePO iteration. Finally, a numerical experiment is given to validate the convergence results. We hope our work paves the way to direct adaptive LQT with online closed-loop data.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑