arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2301.06428math.OCcs.LG
收藏

面向非光滑非凸随机优化的更快无梯度算法

Faster Gradient-Free Algorithms for Nonsmooth Nonconvex Stochastic Optimization

School of Data Science, Fudan University(复旦大学数据科学学院);Institute for Interdisciplinary Information Sciences, Tsinghua University(清华大学交叉信息研究院)

Lesi Chen, Jing Xu, Luo Luo

AI总结:

针对非光滑非凸随机优化问题,采用随机递归梯度估计器,提出了更高效的无梯度算法,降低了求解Goldstein平稳点的随机零阶谕示复杂度。

摘要:

我们研究形如$\min_{x \in \mathbb{R}^d} f(x) \triangleq \mathbb{E}_ξ [F(x; ξ)]$的优化问题,其中分量$F(x;ξ)$满足$L$-均方利普希茨条件,但可能是非凸且非光滑的。近期提出的无梯度方法求解目标函数的$(δ,ε)$-Goldstein平稳点,至多需要$\mathcal{O}( L^4 d^{3/2} ε^{-4} + ΔL^3 d^{3/2} δ^{-1} ε^{-4})$的随机零阶谕示复杂度,其中$Δ= f(x_0) - \inf_{x \in \mathbb{R}^d} f(x)$,且$x_0$为算法的初始点。本文采用随机递归梯度估计器提出了一种更高效的算法,将该复杂度提升至$\mathcal{O}(L^3 d^{3/2} ε^{-3}+ ΔL^2 d^{3/2} δ^{-1} ε^{-3})$。

英文摘要:

We consider the optimization problem of the form $\min_{x \in \mathbb{R}^d} f(x) \triangleq \mathbb{E}_ξ [F(x; ξ)]$, where the component $F(x;ξ)$ is $L$-mean-squared Lipschitz but possibly nonconvex and nonsmooth. The recently proposed gradient-free method requires at most $\mathcal{O}( L^4 d^{3/2} ε^{-4} + ΔL^3 d^{3/2} δ^{-1} ε^{-4})$ stochastic zeroth-order oracle complexity to find a $(δ,ε)$-Goldstein stationary point of objective function, where $Δ= f(x_0) - \inf_{x \in \mathbb{R}^d} f(x)$ and $x_0$ is the initial point of the algorithm. This paper proposes a more efficient algorithm using stochastic recursive gradient estimators, which improves the complexity to $\mathcal{O}(L^3 d^{3/2} ε^{-3}+ ΔL^2 d^{3/2} δ^{-1} ε^{-3})$.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑