arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.01874math.OC
收藏

一种一致自适应的信赖域方法

A consistently adaptive trust-region method

Fadi Hamad, Oliver Hinder

AI总结:

针对现有自适应信赖域方法在 Hessian Lipschitz 常数依赖上次优的问题,提出一种经典信赖域方法的简单自适应变体,达到接近最优的迭代复杂度,并在实验中与 ARC 表现相当。

摘要:

自适应信赖域方法试图在不依赖于对问题性质(例如 Lipschitz 常数)的保守估计的情况下,保持强收敛保证。然而,仔细考察后可以证明,现有自适应信赖域方法的理论保证在对 Hessian 的 Lipschitz 常数等问题性质的依赖上严重次优。例如,Curtin 等人开发的 TRACE 得到了一个 $O(Δ_f L^{3/2} ε^{-3/2}) + \ ilde{O}(1)$ 迭代界,其中 $L$ 是 Hessian 的 Lipschitz 常数。与最优的 $O(Δ_f L^{1/2} ε^{-3/2})$ 界相比,这在关于 $L$ 的方面是次优的。我们提出了第一种规避该问题的自适应信赖域方法,它至多需要 $O(Δ_f L^{1/2} ε^{-3/2}) + \ ilde{O}(1)$ 次迭代来找到一个 $ε$-近似平稳点,在相差一个附加对数项的意义下与最优迭代界相匹配。我们的方法是经典信赖域方法的一个简单变体,并且在我们的实验中与 ARC 和经典信赖域方法相比表现具有竞争力。

英文摘要:

Adaptive trust-region methods attempt to maintain strong convergence guarantees without depending on conservative estimates of problem properties such as Lipschitz constants. However, on close inspection, one can show existing adaptive trust-region methods have theoretical guarantees with severely suboptimal dependence on problem properties such as the Lipschitz constant of the Hessian. For example, TRACE developed by Curtis et al. obtains a $O(Δ_f L^{3/2} ε^{-3/2}) + \tilde{O}(1)$ iteration bound where $L$ is the Lipschitz constant of the Hessian. Compared with the optimal $O(Δ_f L^{1/2} ε^{-3/2})$ bound this is suboptimal with respect to $L$. We present the first adaptive trust-region method which circumvents this issue and requires at most $O( Δ_f L^{1/2} ε^{-3/2}) + \tilde{O}(1)$ iterations to find an $ε$-approximate stationary point, matching the optimal iteration bound up to an additive logarithmic term. Our method is a simple variant of a classic trust-region method and in our experiments performs competitively with both ARC and a classical trust-region method.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑