arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.25643math.OC
收藏

自适应正则化张量方法的局部收敛性

Local Convergence of Adaptively Regularized Tensor Methods

University of Oxford(牛津大学)

Karl Welzel, Yang Liu, Raphael A. Hauser, Coralia Cartis

AI总结:

该研究将正则化张量方法的局部收敛结果扩展到局部一致凸函数与完全自适应方法,首次给出AR$p$的精确局部收敛速率,分析了非凸模型等带来的挑战,证实p足够大时自适应高阶方法在退化问题上可实现超线性收敛并给出精确界。

摘要:

利用目标函数阶数$p>2$的导数的优化方法被称为张量方法。其中,每一步最小化正则化$p$阶泰勒展开的方法已被证明具有最优全局复杂度,且复杂度随$p$增大而提升。Doikov和Nesterov[Math. Program., 193 (2022), pp. 315--336]研究了这类优化算法在具有Lipschitz连续$p$阶导数、且为$q$阶一致凸的函数上的局部收敛性。我们将这些局部收敛结果扩展到局部一致凸函数与无需已知Lipschitz常数的完全自适应方法,从而首次给出了AR$p$的精确局部收敛速率。我们讨论了非凸局部模型和模型极小值不唯一所带来的出人意料的新挑战。对于$p>2$的情况,我们的示例表明,尤其当使用子问题的全局极小值时,即使在渐近意义下,也并非所有迭代都能成功。只有当使用“正确的”局部模型极小值时,非自适应情形下的$p/(q-1)$阶局部收敛性在$p>q-1$时才能保持,否则超线性速率会下降。由此我们证实,只要$p$足够大,自适应高阶方法在某些退化问题上能实现超线性收敛,并给出了极限情况下可预期的收敛阶的精确界。

英文摘要:

Optimization methods that make use of derivatives of the objective function up to order $p > 2$ are called tensor methods. Among them, ones that minimize a regularized $p$th-order Taylor expansion at each step have been shown to possess optimal global complexity, which improves as $p$ increases. The local convergence of such optimization algorithms on functions that have Lipschitz continuous $p$th derivatives and are uniformly convex of order $q$ has been studied by Doikov and Nesterov [Math. Program., 193 (2022), pp. 315--336]. We extend these local convergence results to locally uniformly convex functions and fully adaptive methods, which do not need knowledge of the Lipschitz constant, thus providing the first sharp local rates for AR$p$. We discuss the surprising new challenges encountered by nonconvex local models and non-unique model minimizers. For $p > 2$, our examples show that in particular when using the global minimizer of the subproblem, even asymptotically not all iterations need to be successful. Only if the "right" local model minimizer is used, the $p/(q-1)$th-order local convergence from the non-adaptive case is preserved for $p > q-1$, otherwise the superlinear rate can degrade. We thus confirm that adaptive higher-order methods achieve superlinear convergence for certain degenerate problems as long as $p$ is large enough and provide sharp bounds on the order of convergence one can expect in the limit.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑