通过递归学习和策略梯度实现的稳定性认证在线数据驱动LQR
Stability-Certified On-Policy Data-Driven LQR via Recursive Learning and Policy Gradient
AI总结：
本文提出一种在线数据驱动LQR方法，结合递归最小二乘与策略梯度，通过Lyapunov方法提供稳定性保证，经飞机控制实验验证有效性。
摘要：
本文研究了一种数据驱动框架，用于在动态未知时求解线性二次调节器（LQR）问题，并提供整体学习和控制方案的稳定性证书。所提出的在线学习框架中，控制输入被应用于实际（未知）线性系统并逐步优化。我们提出了一种称为Relearn LQR的学习和控制过程，结合递归最小二乘方法与基于梯度的直接策略搜索。通过将该方案建模为反馈互联的非线性动态系统，利用平均理论和时间尺度分离理论，能够为整个互联方案提供形式稳定性保证。所提策略的有效性通过数值模拟得到验证，其中Relearn LQR被应用于飞行器控制问题，包括静态和漂移参数情况。
英文摘要：
In this paper, we investigate a data-driven framework to solve Linear Quadratic Regulator (LQR) problems when the dynamics is unknown, with the additional challenge of providing stability certificates for the overall learning and control scheme. Specifically, in the proposed on-policy learning framework, the control input is applied to the actual (unknown) linear system while iteratively optimized. We propose a learning and control procedure, termed Relearn LQR, that combines a recursive least squares method with a direct policy search based on the gradient method. The resulting scheme is analyzed by modeling it as a feedback-interconnected nonlinear dynamical system. A Lyapunov-based approach, exploiting averaging and timescale separation theories for nonlinear systems, allows us to provide formal stability guarantees for the whole interconnected scheme. The effectiveness of the proposed strategy is corroborated by numerical simulations, where Relearn LQR is deployed on an aircraft control problem, with both static and drifting parameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
