基于平方和泰勒模型的高阶正则化方法的全局收敛性
Global Convergence of High-Order Regularization Methods with Sums-of-Squares Taylor Models
AI总结：
本文针对非凸光滑优化问题，提出结合平方和（SoS）泰勒模型与自适应正则化的算法框架，证明其在一般非凸和强凸场景下的评估复杂度界，填补了相关算法全局收敛速率分析的空白。
摘要：
近期，针对凸优化与非凸优化问题，研究者提出了在自适应正则化框架内采用基于泰勒的局部模型（次数$p\ge 3$）的高阶张量方法。与牛顿法相比，这类方法已被证明具有更优甚至最优的最坏情况全局收敛速率与局部收敛速率。然而，寻找严谨高效的技术来最小化泰勒多项式子问题，仍是这类算法面临的难点。Ahmadi等人近期提出了一种基于平方和（Sum of Squares，SoS）重构的张量方法，其方法中的每个泰勒多项式子问题都可通过半定规划（SDP）高效求解；但针对一般非凸问题，该方法的全局收敛性与复杂度尚未得到研究。本文提出了一种算法框架，将平方和（SoS）泰勒模型与自适应正则化技术相结合，用于求解非凸光滑优化问题。每次迭代都对一个SoS泰勒模型进行最小化，单次迭代的计算成本为多项式级。针对一般非凸函数，该方法的最坏情况评估复杂度界为$\mathcal{O}(\varepsilon^{-2})$；而针对强凸函数，其评估复杂度界可改进为$\mathcal{O}(\varepsilon^{-\frac{1}{p}})$。据我们所知，这是首次针对非凸光滑优化中具有可解高阶子问题的自适应正则化算法进行全局速率分析，为后续进一步改进开辟了道路。
英文摘要：
High-order tensor methods that employ Taylor-based local models (of degree $p\ge 3$) within adaptive regularization frameworks have been recently proposed for both convex and nonconvex optimization problems. They have been shown to have superior, and even optimal, worst-case global convergence rates and local rates compared to Newton's method. Finding rigorous and efficient techniques for minimizing the Taylor polynomial sub-problems remains a challenging aspect for these algorithms. Ahmadi et al. recently introduced a tensor method based on sum-of-squares (SoS) reformulations, so that each Taylor polynomial sub-problem in their approach can be tractably minimized using semidefinite programming (SDP); however, the global convergence and complexity of their method have not been addressed for general nonconvex problems. This paper introduces an algorithmic framework that combines the Sum of Squares (SoS) Taylor model with adaptive regularization techniques for nonconvex smooth optimization problems. Each iteration minimizes an SoS Taylor model, offering a polynomial cost per iteration. For general nonconvex functions, the worst-case evaluation complexity bound is $\mathcal{O}(ε^{-2})$, while for strongly convex functions, an improved evaluation complexity bound of $\mathcal{O}(ε^{-\frac{1}{p}})$ is established. To the best of our knowledge, this is the first global rate analysis for an adaptive regularization algorithm with a tractable high-order sub-problem in nonconvex smooth optimization, opening the way for further improvements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
