AI 中文总结
本研究证明在非凸-PL 极小极大博弈中,随机梯度下降上升法(SGDA)的复杂度下界与现有上界匹配,且在小时间尺度比例下可能失败,凸显其局限性并支持替代方法。
AI 中文摘要
通过调整时间尺度比例和步长,随机梯度下降上升法(SGDA)在非凸极小极大博弈中能走多远?我们针对非凸-PL(NC-PL)博弈回答了这一问题,首次建立了具有固定时间尺度比例和非递增步长的双时间尺度 SGDA 的紧复杂度。对于具有内部 μ-PL 不等式的 ℓ-光滑博弈,我们证明了复杂度下界 Ω(κ²ℓε⁻²+κ⁴ℓσ²ε⁻⁴),其中 κ=ℓ/μ 为条件数,σ² 为梯度方差,ε 衡量外部梯度范数。这与现有 SGDA 上界匹配,并确立了与 Smoothed-AGDA(Yang 等人,22')的复杂度分离。此外,我们表明当 SGDA 的时间尺度比例小至 o(κ²) 时,它可能无法找到稳定点。我们的负面结果凸显了 SGDA 在 NC-PL 博弈中的根本局限性,并为替代方法的开发提供了依据。
英文摘要
How far can stochastic gradient descent ascent (SGDA) go by tuning its timescale ratio and step sizes in nonconvex min-max games? We answer this question for nonconvex-PL (NC-PL) games by establishing the first tight complexity of two-timescale SGDA with a fixed timescale ratio and non-increasing step sizes. For $\ell$-smooth games with an inner $μ$-PL inequality, we prove a complexity lower bound $Ω(κ^2\ell\varepsilon^{-2}+κ^4\ellσ^2\varepsilon^{-4})$, where $κ=\ell/μ$ is the condition number, $σ^2$ is the gradient variance, and $\varepsilon$ measures the outer gradient norm. This matches existing SGDA upper bounds and establishes a complexity separation from Smoothed-AGDA (Yang et al., 22'). In addition, we show that SGDA can fail to find a stationary point when its timescale ratio is as small as $o(κ^2)$. Our negative results highlight the fundamental limitation of SGDA in NC-PL games, and justify the development of alternative methods.