arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.17488math.OC
收藏

梯度复杂度下凸和非凸优化的更快牛顿法

Faster Newton Methods for Convex and Nonconvex Optimization in Gradient Complexity

IIIS, Tsinghua University(清华大学交叉信息研究院);Shanghai Qizhi Institute(上海齐志研究院);School of Data Science, Fudan University(复旦大学数据科学学院);Shanghai Key Laboratory for Contemporary Applied Mathematics(上海市当代应用数学重点实验室);Shanghai AI Lab(上海人工智能实验室)

Lesi Chen, Chengchang Liu, Luo Luo, Jingzhao Zhang

AI总结:

本文提出新的牛顿方法,改进了非凸和凸优化的梯度复杂度,分别达到O( d̄ + d̄^{1/3} ε^{-3/2})和O( (d̄ + d̄^{13/21} ε^{-2/7}) ln d̄)

摘要:

二阶优化方法对于大规模问题计算成本高昂。Doikov等人提出LazyCRN方法,其非凸和凸优化的梯度复杂度分别为$\mathcal{O}( \bar d + \bar d^{1/2} \epsilon^{-3/2})$和$\mathcal{O}( \bar d + \bar d^{1/2} \epsilon^{-1/2})$。Adil等人改进了非凸优化的梯度复杂度。本文提出新方法,非凸和凸优化的复杂度分别为$\mathcal{O}( \bar d + \bar d^{1/3} \epsilon^{-3/2})$和$\mathcal{O}( (\bar d + \bar d^{13/21} \epsilon^{-2/7}) \ln \bar d)$,改进了现有最佳结果。

英文摘要:

Second-order optimization methods are computationally expensive for large-scale problems. Recently, Doikov, Chayti, and Jaggi (ICML 2023) proposed the LazyCRN method that reduces computation by studying the gradient complexity of second-order methods. Their method can achieve a gradient complexity of $\mathcal{O}( \bar d + \bar d^{1/2} ε^{-3/2})$ and $\mathcal{O}( \bar d + \bar d^{1/2} ε^{-1/2})$ for nonconvex and convex optimization, respectively, where $\bar d$ is the effective dimension and $ε$ is the target precision. Very recently, Adil, Bullins, Sidford, and Zhang (NeurIPS 2025) improved the gradient complexity to $\mathcal{O}( \bar d + \bar d^{1/3} ε^{-3/2} \ln^{18} ε^{-1})$ for nonconvex optimization. However, the tightness of these methods remains open. In this work, we propose new methods that achieve an improved complexity of $\mathcal{O}( \bar d + \bar d^{1/3} ε^{-3/2})$ and $\mathcal{O}( (\bar d + \bar d^{13/21} ε^{-2/7}) \ln \bar d)$ for nonconvex and convex optimization, respectively, improving best-known results for both setups.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑