单循环随机投影阻尼外梯度方法用于随机非凸--（强）凹极小极大优化
Single-Loop Stochastic Projected Damped Extragradient Methods for Stochastic Nonconvex--(Strongly) Concave Minimax Optimization
- Shanghai University（上海大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文提出单循环随机投影阻尼外梯度方法及其方差缩减变体，用于求解随机非凸--（强）凹极小极大问题，在博弈和优化平稳性下均达到最佳已知的随机一阶预言复杂度。
AI中文摘要：
我们针对随机非凸--（强）凹极小极大优化问题开发了单循环随机投影阻尼外梯度方法，并给出了博弈平稳性（GS）和优化平稳性（OS）的复杂度保证。我们的方法结合了随机投影阻尼外梯度（SPDE）方法及其递归方差缩减变体VR-SPDE，两者均保持单循环结构。在具有一致有界方差的非偏随机梯度预言下，SPDE在非凸--强凹和非凸--凹设置下分别以$O(\kappa\varepsilon^{-4})$和$O(\varepsilon^{-5})$的随机一阶预言（SFO）复杂度找到$\varepsilon$-博弈平稳点，其中$\kappa=L/\mu$。在随机梯度满足额外均方Lipschitz条件下，VR-SPDE将GS复杂度分别提升至$O(\kappa^{3/2}\varepsilon^{-3})$和$O(\varepsilon^{-9/2})$。对于$\varepsilon$-优化平稳点，SPDE在两种设置下分别达到$O(\kappa\varepsilon^{-4})$和$O(\varepsilon^{-6})$的SFO复杂度，而VR-SPDE分别达到$O(\kappa^{3/2}\varepsilon^{-3})$和$O(\varepsilon^{-6})$。这些OS保证与多循环方法所达到的最佳已知界相匹配，同时保持单循环实现。据我们所知，我们的结果为各自平稳性准则和问题类别下的单循环随机一阶方法提供了最佳已知的SFO复杂度保证。
英文摘要：
We develop single-loop stochastic projected damped extragradient methods for stochastic nonconvex--(strongly) concave minimax optimization, with complexity guarantees for both game stationarity (GS) and optimization stationarity (OS). Our approach combines a stochastic projected damped extragradient (SPDE) method with a recursive variance-reduced variant, VR-SPDE, both of which retain a single-loop structure. Under an unbiased stochastic gradient oracle with uniformly bounded variance, SPDE finds an $\varepsilon$-game-stationary point with stochastic first-order oracle (SFO) complexities of $O(κ\varepsilon^{-4})$ and $O(\varepsilon^{-5})$ in the nonconvex--strongly concave and nonconvex--concave settings, respectively, where $κ=L/μ$. Under an additional mean-square Lipschitz condition on the stochastic gradients, VR-SPDE improves these GS complexities to $O(κ^{3/2}\varepsilon^{-3})$ and $O(\varepsilon^{-9/2})$, respectively. For an $\varepsilon$-optimization-stationary point, SPDE achieves SFO complexities of $O(κ\varepsilon^{-4})$ and $O(\varepsilon^{-6})$, while VR-SPDE achieves $O(κ^{3/2}\varepsilon^{-3})$ and $O(\varepsilon^{-6})$, in the two settings, respectively. These OS guarantees match the best-known bounds achieved by multi-loop methods while preserving a single-loop implementation. To the best of our knowledge, our results provide the best-known SFO complexity guarantees among single-loop stochastic first-order methods for the respective stationarity criteria and problem classes.
