非精确梯度估计和动态Hessian精度下自适应三次正则化方法的随机分析
Stochastic Analysis of an Adaptive Cubic Regularisation Method under Inexact Gradient Evaluations and Dynamic Hessian Accuracy
摘要：
在这里，我们将[3]中用于非凸优化的具有动态不精确Hessian信息的自适应三次正则化方法的扩展版本适应于随机优化设置。虽然仍然考虑精确的函数评估，但这种新的变体继承了[3]中引入的Hessian近似的自适应精度要求的创新使用，并且还采用了梯度的不精确计算。在不限制误差方差的情况下，我们假设这些近似在足够大但固定的概率内可用，并且我们本着[18]的精神将框架的确定性分析扩展到其随机对应物，表明达到一阶稳定点的预期迭代次数与众所周知的最差情况最优复杂度相匹配。实际上，对于一阶误差容差，这仍然由O（θ ^（-3/2））给出。最后，非凸有限和最小化的数值试验证实，使用不精确的一阶和二阶导数可以是有益的计算节省。
英文摘要：
We here adapt an extended version of the adaptive cubic regularisation method with dynamic inexact Hessian information for nonconvex optimisation in [3] to the stochastic optimisation setting. While exact function evaluations are still considered, this novel variant inherits the innovative use of adaptive accuracy requirements for Hessian approximations introduced in [3] and additionally employs inexact computations of the gradient. Without restrictions on the variance of the errors, we assume that these approximations are available within a sufficiently large, but fixed, probability and we extend, in the spirit of [18], the deterministic analysis of the framework to its stochastic counterpart, showing that the expected number of iterations to reach a first-order stationary point matches the well known worst-case optimal complexity. This is, in fact, still given by O(epsilon^(-3/2)), with respect to the first-order epsilon tolerance. Finally, numerical tests on nonconvex finite-sum minimisation confirm that using inexact first and second-order derivatives can be beneficial in terms of the computational savings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
