arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.02140math.OCcs.SYeess.SY
收藏

关于LQR问题中超参数化策略优化的(几乎)全局指数收敛性

On the (almost) Global Exponential Convergence of the Overparameterized Policy Optimization for the LQR Problem

Northeastern University(东北大学)

Moh Kamalul Wafi, Arthur Castello B. de Oliveira, Eduardo D. Sontag

AI总结:

本研究针对连续时间LQR问题的策略优化,提出通过LFFNN超参数化的方法,证明其可实现几乎全局指数收敛,且该提升在一般场景下经数值验证成立。

摘要:

在本研究中,我们探讨了非凸优化问题下梯度方法的收敛性——具体而言,是问题建模方式对梯度流解收敛行为的影响。我们通过一个简单示例发现,出人意料的是,梯度流解既可能呈指数收敛,也可能呈渐近收敛,这取决于问题的建模方式。随后我们深化分析,证明针对连续时间线性二次调节器(LQR,已知其全局仅呈现渐近收敛)的策略优化策略,若通过线性前馈神经网络(LFFNN)对问题进行超参数化处理,将呈现几乎全局指数收敛性。我们证实,对于简化版LQR问题,这种质性提升始终存在,并推导了梯度流的显式收敛速率。最后,我们通过数值模拟表明,在一般LQR问题中,这种质性提升与量化速率增益均持续存在。

英文摘要:

In this work we study the convergence of gradient methods for nonconvex optimization problems -- specifically the effect of the problem formulation to the convergence behavior of the solution of a gradient flow. We show through a simple example that, surprisingly, the gradient flow solution can be exponentially or asymptotically convergent, depending on how the problem is formulated. We then deepen the analysis and show that a policy optimization strategy for the continuous-time linear quadratic regulator (LQR) (which is known to present only asymptotic convergence globally) presents almost global exponential convergence if the problem is overparameterized through a linear feed-forward neural network (LFFNN). We prove this qualitative improvement always happens for a simplified version of the LQR problem and derive explicit convergence rates for the gradient flow. Finally, we show that both the qualitative improvement and the quantitative rate gains persist in the general LQR through numerical simulations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑