arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1808.02871math.OCcs.LG
收藏

确定性扰动下的随机方向随机逼近

Random directions stochastic approximation with deterministic perturbations

Indian Institute of Technology Madras(印度理工学院马德拉斯分校);Indian Institute of Science(印度科学研究所);University of Maryland(马里兰大学)

Prashanth L A, Shalabh Bhatnagar, Nirav Bhavsar, Michael Fu, Steven I. Marcus

摘要:

我们为最近提出的随机方向随机逼近(RDSA)[17]引入了确定性扰动方案,并提出了新的一阶和二阶算法。在后一种情况下,这些是第一个二阶算法,将确定性扰动。我们证明了在确定性扰动下的算法中的梯度和/或Hessian估计是渐近无偏的,因此该算法是可证明收敛的。此外,我们推导出收敛速度建立的一阶和二阶算法的优越性,为特殊情况下的凸和二次最优化问题,分别。数值实验验证了理论结果。

英文摘要:

We introduce deterministic perturbation schemes for the recently proposed random directions stochastic approximation (RDSA) [17], and propose new first-order and second-order algorithms. In the latter case, these are the first second-order algorithms to incorporate deterministic perturbations. We show that the gradient and/or Hessian estimates in the resulting algorithms with deterministic perturbations are asymptotically unbiased, so that the algorithms are provably convergent. Furthermore, we derive convergence rates to establish the superiority of the first-order and second-order algorithms, for the special case of a convex and quadratic optimization problem, respectively. Numerical experiments are used to validate the theoretical results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑