通过输出变量实现深度和循环神经网络的安全变异
Safe Mutations for Deep and Recurrent Neural Networks through Output Gradients
摘要:
虽然神经进化(进化神经网络)在从强化学习到人工生命的各个领域都有成功的记录,但它很少应用于大型深层神经网络。一个核心原因是,虽然随机突变通常在低维中工作,但数千或数百万个权重的随机扰动可能会破坏现有功能,即使某些个别权重变化是有益的,也不会提供学习信号。本文提出了一种解决方案,引入了一系列安全变异(SM)算子,旨在变异算子本身找到一个不会改变网络行为太多的变化程度,但仍然有利于探索。重要的是,这些SM操作符不需要与环境进行任何额外的交互。最有效的SM变体利用了一个有趣的机会,根据网络输出对每个权重的敏感度来缩放每个权重的突变程度,这需要计算输出相对于权重的梯度(而不是传统深度学习中的误差梯度)。这种通过梯度的安全突变(SM-G)算子极大地提高了基于简单遗传算法的神经进化方法在需要深度和/或循环神经网络(其倾向于特别容易突变)的高维域中找到解决方案的能力,包括需要处理原始像素的域。通过提高我们进化深层神经网络的能力,这种新的更安全的突变方法扩大了适合神经进化的领域的范围。
英文摘要:
While neuroevolution (evolving neural networks) has a successful track record across a variety of domains from reinforcement learning to artificial life, it is rarely applied to large, deep neural networks. A central reason is that while random mutation generally works in low dimensions, a random perturbation of thousands or millions of weights is likely to break existing functionality, providing no learning signal even if some individual weight changes were beneficial. This paper proposes a solution by introducing a family of safe mutation (SM) operators that aim within the mutation operator itself to find a degree of change that does not alter network behavior too much, but still facilitates exploration. Importantly, these SM operators do not require any additional interactions with the environment. The most effective SM variant capitalizes on the intriguing opportunity to scale the degree of mutation of each individual weight according to the sensitivity of the network's outputs to that weight, which requires computing the gradient of outputs with respect to the weights (instead of the gradient of error, as in conventional deep learning). This safe mutation through gradients (SM-G) operator dramatically increases the ability of a simple genetic algorithm-based neuroevolution method to find solutions in high-dimensional domains that require deep and/or recurrent neural networks (which tend to be particularly brittle to mutation), including domains that require processing raw pixels. By improving our ability to evolve deep neural networks, this new safer approach to mutation expands the scope of domains amenable to neuroevolution.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。