arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.01647math.OCcs.CC
收藏

一类无目标函数一阶优化算法的复杂度

Complexity of a Class of First-Order Objective-Function-Free Optimization Algorithms

S. Gratton, S. Jerad, Ph. L. Toint

AI总结:

研究无需计算目标函数值的信赖域优化算法,证明其复杂度可媲美使用函数值与梯度的一阶方法,并提出一类本质最优的新算法。

摘要:

本文考虑用于无约束非凸优化的一类参数化信赖域算法,其中从不计算目标函数值。该类别包含通常用于最小化含噪声函数的一阶 Adagrad 方法的确定性版本,但也允许在二阶信息可用时使用该信息(可以是近似的)。文中分析了该类别中方法的收敛速率,并证明其与已知的同时使用函数值和梯度值的一阶优化方法的收敛速率相同,从而恢复了纯一阶变体的已有结果,并改进了对问题维度的显式依赖。该速率被证明本质上是最优的。文中还提出了一类新方法,对其成立一个稍差但本质上最优的复杂度结果。有限的数值实验表明,尽管使用的信息显著更少,新方法的性能仍可与标准最速下降相媲美,而且该性能对噪声相对不敏感。

英文摘要:

A parametric class of trust-region algorithms for unconstrained nonconvex optimization is considered where the value of the objective function is never computed. The class contains a deterministic version of the first-order Adagrad method typically used for minimization of noisy function, but also allows the use of (possibly approximate) second-order information when available. The rate of convergence of methods in the class is analyzed and is shown to be identical to that known for first-order optimization methods using both function and gradients values, recovering existing results for purely-first order variants and improving the explicit dependence on problem dimension. This rate is shown to be essentially sharp. A new class of methods is also presented, for which a slightly worse and essentially sharp complexity result holds. Limited numerical experiments show that the new methods' performance may be comparable to that of standard steepest descent, despite using significantly less information, and that this performance is relatively insensitive to noise.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑