非光滑非凸优化中寻找驻点的复杂性
The Complexity of Finding Stationary Points in Nonsmooth Nonconvex Optimization
浏览论文内容
中文总结 AI 辅助
本文证明非光滑非凸优化中一阶算法寻找 Goldstein 驻点及松弛驻点的紧下界，揭示收敛速率不受梯度随机性影响。
中文摘要 AI 辅助
我们证明，一阶算法在最坏情况下需要 Ω(δ⁻¹ε⁻³) 次梯度查询才能找到 Lipschitz 函数的 (δ,ε)-Goldstein 驻点，在该点处存在一个距离在 δ 内的梯度的凸组合，其范数至多为 ε。该下界是紧的，与已知算法匹配至绝对常数，从而解决了非光滑非凸优化中收敛到驻点的复杂性。我们进一步证明了 Ω(λ^{1/2}ε^{-7/2}) 的紧下界，用于寻找满足最近提出的 (λ,ε)-驻点松弛概念的点，该概念允许组合更远的梯度。我们的结果表明，与非光滑优化相比，梯度随机性不影响非光滑驻点的收敛速率，这与光滑优化形成鲜明对比。
英文摘要
We prove that first-order algorithms require $Ω(δ^{-1}ε^{-3})$ gradient queries (in the worst case) to find a $(δ,ε)$-Goldstein stationary point of a Lipschitz function, at which there is a convex combination of gradients within distance $δ$ whose norm is at most $ε$. This lower bound is tight, matching known algorithms up to absolute constants, therefore resolving the complexity of convergence to stationarity in nonsmooth nonconvex optimization. We further prove a tight lower bound of $Ω(λ^{1/2}ε^{-7/2})$ for finding points satisfying the recently proposed relaxed notion of $(λ,ε)$-stationarity, which allows combining further-away gradients. Our results reveal that convergence rates to nonsmooth stationarity are not affected by gradient stochasticity, in sharp contrast to smooth optimization.
