噪声基线下节点扰动学习的统计机制
Statistical Mechanics of Node-perturbation Learning with Noisy Baseline
摘要:
节点扰动学习是一种统计梯度下降算法,可以应用于目标函数没有显式公式化的问题,包括强化学习。它通过使用响应于扰动的目标函数的变化来估计目标函数的梯度。未扰动输出的目标函数的值称为基线。Cho等人提出了带噪声基线的节点扰动学习。在本文中,我们报告的统计力学的赵的模型和推导耦合微分方程的序参数,描绘学习动态。我们还展示了如何通过求解阶参数微分方程来推导推广误差。在此基础上,我们证明了Cho的结果在一般情况下也是适用的,并展示了Cho模型的一些一般性质。
英文摘要:
Node-perturbation learning is a type of statistical gradient descent algorithm that can be applied to problems where the objective function is not explicitly formulated, including reinforcement learning. It estimates the gradient of an objective function by using the change in the object function in response to the perturbation. The value of the objective function for an unperturbed output is called a baseline. Cho et al. proposed node-perturbation learning with a noisy baseline. In this paper, we report on building the statistical mechanics of Cho's model and on deriving coupled differential equations of order parameters that depict learning dynamics. We also show how to derive the generalization error by solving the differential equations of order parameters. On the basis of the results, we show that Cho's results are also apply in general cases and show some general performances of Cho's model.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。