arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2008.06643cs.NEcond-mat.stat-mech
收藏

神经进化与梯度下降的对应关系

Correspondence between neuroevolution and gradient descent

Stephen Whitelam, Viktor Selin, Sang-Won Park, Isaac Tamblyn

摘要:

我们分析表明,训练神经网络的条件随机突变或神经进化的权重是等价的,在小突变的限制,梯度下降的损失函数在高斯白噪声的存在下。在学习过程的独立实现上求平均,神经进化等价于损失函数上的梯度下降。我们使用数值模拟表明,这种对应关系可以观察到有限的突变,浅和深神经网络。我们的研究结果提供了两种神经网络训练方法之间的联系,这两种方法通常被认为是根本不同的。

英文摘要:

We show analytically that training a neural network by conditioned stochastic mutation or neuroevolution of its weights is equivalent, in the limit of small mutations, to gradient descent on the loss function in the presence of Gaussian white noise. Averaged over independent realizations of the learning process, neuroevolution is equivalent to gradient descent on the loss function. We use numerical simulation to show that this correspondence can be observed for finite mutations,for shallow and deep neural networks. Our results provide a connection between two families of neural-network training methods that are usually considered to be fundamentally different.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑