arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1405.0769stat.ME
收藏

同时摄动随机逼近中非Bernoulli小样本摄动分布

Non-Bernoulli Perturbation Distributions for Small Samples in Simultaneous Perturbation Stochastic Approximation

Xumeng Cao

摘要:

同时扰动随机逼近(SPSA)已被证明是有效的递归优化。SPSA基于两个函数评估对梯度使用中心差分近似,而不管问题的维度如何。通常,伯努利+-1分布用于扰动向量,并且已经建立了理论来证明该分布的渐近最优性。然而,伯努利分布的最优性可能不适用于小样本随机逼近(SA)运行。在本文中,我们研究的性能分段均匀扰动分布的小样本SPSA。特别是,我们进行了理论分析SA的一次迭代,这是一个合理的起点,可以作为推广到其他小样本SPSA设置与一个以上的迭代的基础。在这项工作中,我们表明,伯努利分布可能不是最好的选择扰动向量在某些参数的选择小样本SPSA

英文摘要:

Simultaneous perturbation stochastic approximation (SPSA) has proven to be efficient for recursive optimization. SPSA uses a centered difference approximation to the gradient based on two function evaluations regardless of the dimension of the problem. Typically, the Bernoulli +-1 distribution is used for perturbation vectors and theory has been established to prove the asymptotic optimality of this distribution. However, optimality of the Bernoulli distribution may not hold for small-sample stochastic approximation (SA) runs. In this paper, we investigate the performance of the segmented uniform as a perturbation distribution for small-sample SPSA. In particular, we conduct a theoretical analysis for one iteration of SA, which is a reasonable starting point and can be used as a basis for generalization to other small-sample SPSA settings with more than one iteration. In this work, we show that the Bernoulli distribution may not be the best choice for perturbation vectors under certain choices of parameters in small-sample SPSA

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑