线搜索正则化的随机拟牛顿法
Stochastic quasi-Newton with line-search regularization
摘要：
在本文中，我们提出了一种新的拟牛顿算法用于随机优化。拟牛顿方法对确定性优化问题产生了巨大的影响，因为它们提供了快速收敛和计算上有吸引力的算法。本质上，这是通过基于观察一阶梯度来学习二阶（Hessian）信息来实现的。我们将这些想法扩展到随机设置，通过采用高度灵活的海森模型，并根据观察噪声梯度推断其值。此外，我们提出了一个随机对应的标准线搜索程序，并证明了这种组合的效用一般非线性状态空间模型的最大似然识别。
英文摘要：
In this paper we present a novel quasi-Newton algorithm for use in stochastic optimisation. Quasi-Newton methods have had an enormous impact on deterministic optimisation problems because they afford rapid convergence and computationally attractive algorithms. In essence, this is achieved by learning the second-order (Hessian) information based on observing first-order gradients. We extend these ideas to the stochastic setting by employing a highly flexible model for the Hessian and infer its value based on observing noisy gradients. In addition, we propose a stochastic counterpart to standard line-search procedures and demonstrate the utility of this combination on maximum likelihood identification for general nonlinear state space models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
