在 $(\alpha,L_0,L_1)$-光滑性下的信赖域方法复杂度分析
Complexity analysis of trust-region methods under $(α,L_0,L_1)$-smoothness
- Polytechnique Montréal（蒙特利尔理工大学）
- GERAD
- CIRRELT
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文证明信赖域方法在无需参数先验知识的情况下，对$(\alpha,L_0,L_1)$-光滑目标函数达到最佳已知复杂度界，并支持一般Hessian近似，且界在非凸情形下是紧的。
AI中文摘要：
广义光滑性假设近年来受到越来越多的关注，部分原因是机器学习问题中目标函数的梯度可能不是Lipschitz连续的。其中最突出的是 $(\alpha, L_0, L_1)$-光滑性假设。现有的达到最佳已知复杂度界的方法需要知道或上界 $\alpha$、$L_0$ 和 $L_1$，而少数可用的参数无关方法无法恢复这些界。在本文中，我们表明信赖域方法在无需事先知道这些参数的情况下，对于 $(\alpha, L_0, L_1)$-光滑目标函数能达到最佳已知界。建立这些结果需要超越经典信赖域复杂度分析中使用的新分析工具。我们进一步表明，我们的工作模型假设允许使用一般的模型Hessian近似，特别是可以容纳有限内存的拟牛顿更新。最后，我们证明我们的复杂度界在非凸情形下是紧的。
英文摘要：
Generalized smoothness assumptions have attracted growing attention in recent years, motivated in part by machine learning problems in which the gradient of the objective function may not be Lipschitz continuous. Among the most prominent of these is the $(α, L_0, L_1)$-smoothness assumption. Existing methods that attain the best known complexity bounds require knowledge of, or upper bounds on, $α$, $L_0$ and $L_1$, while the few parameter-agnostic methods available do not recover those bounds. In this paper, we show that trust-region methods attain the best known bounds for $(α, L_0, L_1)$-smooth objective functions without prior knowledge of these parameters. Establishing these results requires new analytical tools, beyond those used in classical trust-region complexity analyses. We further show that our working model assumption allows the use of general model Hessian approximations and, in particular, accommodates limited-memory quasi-Newton updates. Finally, we show that our complexity bound is sharp in the nonconvex setting.
