Polyak-Łojasiewicz条件下双层学习的广义交替方法
A Generalized Alternating Method for Bilevel Learning under the Polyak-Łojasiewicz Condition
AI总结：
针对下层非凸且满足PL条件的双层优化问题，提出广义交替方法GALET，证明其迭代复杂度与单层光滑非凸问题的梯度下降相当，拓展了双层优化交替算法的适用场景。
摘要：
双层优化近期因在超参数优化、元学习、强化学习等新兴机器学习领域的应用重新受到关注。已有研究表明，在求解下层目标为强凸的双层问题时，简单的基于交替（隐式）梯度的算法可达到与单层梯度下降（GD）相当的收敛速率。但目前尚不清楚该结果能否推广到这一基本设定之外的双层问题。本文首先针对满足Polyak-Łojasiewicz（PL）条件的非凸下层目标，为所研究的双层问题引入了一种平稳性度量，该度量推广了现有度量。随后我们提出了面向双层优化的广义交替方法（Generalized ALternating mEthod for bilevel opTimization, GALET），该方法专为下层问题为凸PL问题的双层优化（BLO）设计，并证明GALET可在$\tilde{\boldsymbol{\rm O}}(\boldsymbol{\rm ε}^{-1})$次迭代内得到所研究问题的$\boldsymbol{\rm ε}$-平稳点，这一迭代复杂度与单层光滑非凸问题的梯度下降（GD）迭代复杂度相当。
英文摘要：
Bilevel optimization has recently regained interest owing to its applications in emerging machine learning fields such as hyperparameter optimization, meta-learning, and reinforcement learning. Recent results have shown that simple alternating (implicit) gradient-based algorithms can match the convergence rate of single-level gradient descent (GD) when addressing bilevel problems with a strongly convex lower-level objective. However, it remains unclear whether this result can be generalized to bilevel problems beyond this basic setting. In this paper, we first introduce a stationary metric for the considered bilevel problems, which generalizes the existing metric, for a nonconvex lower-level objective that satisfies the Polyak-Łojasiewicz (PL) condition. We then propose a Generalized ALternating mEthod for bilevel opTimization (GALET) tailored to BLO with convex PL LL problem and establish that GALET achieves an $ε$-stationary point for the considered problem within $\tilde{\cal O}(ε^{-1})$ iterations, which matches the iteration complexity of GD for single-level smooth nonconvex problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
