arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.09947math.OC
收藏

一个目标函数Optimization正则化算法的收敛性及$\mathcal{O}(ε^{-3/2})$复杂性界

Convergence properties of an Objective-Function-Free Optimization regularization algorithm, including an $\mathcal{O}(ε^{-3/2})$ complexity bound

S. Gratton, S. Jerad, Ph. L. Toint

摘要:

提出了一种求解无约束非凸优化问题的自适应正则化算法。该算法属于自适应正则化方法的类,对于其中评估目标函数的标准框架,已知最佳最坏情况复杂度结果。它表明,在本文中,这些优秀的复杂性界限也是有效的新算法,尽管事实上,显着较少的信息被使用。特别是,它示出,如果衍生物的程度$p$使用,该算法将找到一个$ε_1$近似的一阶极小在最多的$O(ε_1^{-(p+1)/p})$迭代,和一个$(ε_1,ε_2)$近似的二阶极小在最多的$O(\max[ε^{-(p+1)/p},ε_2^{-(p+1)/(p-1)}])$迭代。作为一种特殊情况,使用一阶和二阶导数的新算法,当应用于具有Lipschitz连续Hessian的函数时,将在至多$O(ε_1^{-3/2})$迭代中找到梯度范数小于$ε_1$的$x_k$。

英文摘要:

An adaptive regularization algorithm for unconstrained nonconvex optimization is presented in which the objective function is never evaluated, but only derivatives are used. This algorithm belongs to the class of adaptive regularization methods, for which optimal worst-case complexity results are known for the standard framework where the objective function is evaluated. It is shown in this paper that these excellent complexity bounds are also valid for the new algorithm, despite the fact that significantly less information is used. In particular, it is shown that, if derivatives of degree one to $p$ are used, the algorithm will find a $ε_1$-approximate first-order minimizer in at most $O(ε_1^{-(p+1)/p})$ iterations, and an $(ε_1,ε_2)$-approximate second-order minimizer in at most $O(\max[ε^{-(p+1)/p},ε_2^{-(p+1)/(p-1)}])$ iterations. As a special case, the new algorithm using first and second derivatives, when applied to functions with Lipschitz continuous Hessian, will find an iterate $x_k$ at which the gradient's norm is less than $ε_1$ in at most $O(ε_1^{-3/2})$ iterations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑