低秩分解是深度神经进化的间接编码
Low Rank Factorizations are Indirect Encodings for Deep Neuroevolution
AI总结:
本文提出低秩分解神经进化方法,通过搜索小规模低秩因子空间来强制网络权重结构,在语言建模和视觉强化学习任务中表现优异,并证明有害分解突变能显著减少运行时间。
摘要:
深度神经进化是强化学习的一种高度可扩展的替代方案,因其独特的能力能够以少量字节编码网络更新。来自传统深度学习的最新见解表明,高维模型具有内在的低秩结构。在这项工作中,我们引入了低秩、分解的神经进化:一种间接编码,通过它我们可以搜索一个小的低秩因子空间,这些因子在网络权重之间强制执行底层结构。我们将我们的方法与相似和更小尺寸的非分解网络进行比较,以了解有多少性能可以归因于更小的搜索空间。我们在使用transformers的语言建模任务以及连续和离散的基于视觉的强化学习任务上评估我们的方法。我们的研究表明,低秩、分解的神经进化优于或与非分解神经进化具有竞争力,在语言建模上表现尤为出色。我们的结果还表明,有害的分解突变对性能的负面影响比有害的非分解突变更强,这显著减少了在具有对不良表现者提前终止的环境中运行时间。更广泛地说,这些结果显示了我们如何使用来自基于反向传播方法的见解来增强神经进化。
英文摘要:
Deep neuroevolution is a highly scalable alternative to reinforcement learning due to its unique ability to encode network updates in a small number of bytes. Recent insights from traditional deep learning indicate high-dimensional models possess intrinsic, low-rank structure. In this work, we introduce low-rank, factorized neuroevolution: an indirect encoding through which we can search a small space of low-rank factors that enforce underlying structure across a network's weights. We compare our approach with non-factorized networks of similar and smaller size to understand how much performance can be attributed to the smaller search space. We evaluate our method on a language modeling task using transformers, as well as continuous and discrete vision-based reinforcement learning tasks. Our study shows that low-rank, factorized neuroevolution outperforms or is competitive with non-factorized neuroevolution, performing notably well on language modeling. Our results also suggest deleterious factorized mutations have a stronger negative impact on performance than deleterious non-factorized mutations, which significantly reduces the runtime on environments with early termination for bad performers. More broadly, these results show how we can use insights from backpropgation-based methods to enhance neuroevolution
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。