求解具有非凸下层问题的双层优化需要二阶平稳性

中文摘要 AI 辅助

尽管双层优化（BLO）近年来已成为解决许多复杂嵌套机器学习问题的强大框架，但现有研究大多局限于下层强凸（LLSC）或下层一般凸（LLGC）设置（即下层目标函数至少被假定为凸的）。虽然LLSC/LLGC假设使算法设计和理论分析更加易处理，但它们过于严格，无法涵盖实践中许多机器学习问题。BLO中LLSC/LLGC假设的局限性促使我们研究在一般下层非凸（LLNC）设置下求解BLO问题，这一领域目前仍处于起步阶段。在LLNC-BLO的文献中，现有工作要么要求下层目标函数具有额外结构以便进行易处理的理论分析，要么采用从LLSC/LLGC设置继承而来的一阶平稳性重构作为下层替代问题，但后者在LLNC设置中可能失效。为弥合这一差距，我们提出使用基于二阶平稳性的替代问题来重构非凸下层问题，其解保证下层局部最优解。基于此重构，我们提出了PROBE（用于双层问题的扰动梯度算法），并证明它通过探测和逃逸下层鞍点克服了先前工作的局限性。我们证明PROBE实现了$O(T^{-2/5})$的有限时间收敛速率，其中T表示迭代次数。据我们所知，这项工作首次在一般LLNC-BLO中建立了实现下层二阶平稳解的有限时间收敛性。我们在基于大语言模型的数据整理任务和元学习任务上的实验也表明，PROBE优于最先进的方法。

英文摘要

Although bilevel optimization (BLO) has emerged as a powerful framework for addressing many complex and nested machine learning problems in recent years, most existing studies are confined to the lower-level strongly convex (LLSC) or lower-level generally convex (LLGC) settings (i.e., the lower-level objective function is assumed to be, at least, convex). While the LLSC/LLGC assumptions render more tractable algorithmic design and theoretical analysis, they are too rigid to encompass many machine learning problems in practice. The limitations of LLSC/LLGC assumptions in BLO motivate us to investigate solving the BLO problem in the general lower-level nonconvex (LLNC) settings, which remains in its infancy. In the literature on LLNC-BLO, most of the existing works either require additional structures in the lower-level objective function for tractable theoretical analysis, or adopt the first-order stationarity reformulation as a lower-level surrogate problem, which is inherited from the LLSC/LLGC settings but could lose their effectiveness in the LLNC setting. To bridge this gap, we propose to reformulate the nonconvex lower-level problem using a second-order stationarity-based surrogate, the solution of which guarantees a local optimal solution at the lower level. Based on this reformulation, we propose the PROBE (Perturbed gradient algorithm for bilevel problem) and show that it overcomes the limitations of prior works by probing and escaping lower-level saddle points. We prove that PROBE achieves a finite-time convergence rate of $O(T^{-2/5})$, where T denotes iterations. To our knowledge, this work is the first to establish the finite-time convergence for achieving lower-level second-order stationary solutions in general LLNC-BLO. Our experiments on both a large language model-based data curation task and a meta-learning task also show that PROBE outperforms SOTA methods.

发表机构

• The Ohio State University（俄亥俄州立大学）
• Meta
• University of Colorado Boulder（科罗拉多大学博尔德分校）
• Johns Hopkins University（约翰斯·霍普金斯大学）

机构由 AI 辅助整理，请以论文原文为准。