arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1802.06925math.OC
收藏

不精确非凸Newton型方法

Inexact Non-Convex Newton-Type Methods

Zhewei Yao, Peng Xu, Farbod Roosta-Khorasani, Michael W. Mahoney

摘要:

为了解决大规模非凸问题,我们提出了置信域和自适应立方正则化方法的不精确变体,其中,为了提高效率,结合各种近似。特别地,除了近似子问题解决之外,Hessian和梯度都被适当地近似。使用相当温和的条件,这样的近似,我们表明,我们提出的不精确的方法实现类似的最佳最坏情况下的迭代复杂性的确切的同行。我们提出的算法,以及它们各自的理论分析,不需要任何不可知的问题相关的量的知识,因此很容易在实践中实现。在有限和问题的背景下,我们探索随机子采样方法作为构建梯度和Hessian近似的方法,并在一些真实数据集上检验我们的算法的经验性能。

英文摘要:

For solving large-scale non-convex problems, we propose inexact variants of trust region and adaptive cubic regularization methods, which, to increase efficiency, incorporate various approximations. In particular, in addition to approximate sub-problem solves, both the Hessian and the gradient are suitably approximated. Using rather mild conditions on such approximations, we show that our proposed inexact methods achieve similar optimal worst-case iteration complexities as the exact counterparts. Our proposed algorithms, and their respective theoretical analysis, do not require knowledge of any unknowable problem-related quantities, and hence are easily implementable in practice. In the context of finite-sum problems, we then explore randomized sub-sampling methods as ways to construct the gradient and Hessian approximations and examine the empirical performance of our algorithms on some real datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑