arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1803.04439cs.NEcs.LG
收藏

从节点到网络:进化的循环神经网络

From Nodes to Networks: Evolving Recurrent Neural Networks

Sentient Technologies, Inc(森蒂恩特科技公司)

Aditya Rawal, Risto Miikkulainen

摘要:

诸如由长短期记忆(LSTM)节点组成的那些门控递归网络最近已被用于改进诸如语音识别和机器翻译的许多顺序处理任务中的现有技术。然而,LSTM节点的基本结构与25年前首次构想时基本相同。最近,进化和强化学习机制已被用于创建这种结构的新变体。本文提出了一种新的方法,进化树为基础的编码的门控记忆节点,并表明,它可以探索新的变化比其他方法更有效。该方法发现了具有多个递归路径和多个存储单元的节点,这导致了标准语言建模基准任务的显著改进。本文还展示了如何通过训练LSTM网络来评估候选结构的性能,以及鼓励探索新的解决方案来加快搜索过程。因此,复杂神经网络结构的进化设计有望提高深度学习架构的性能,超出人类的能力。

英文摘要:

Gated recurrent networks such as those composed of Long Short-Term Memory (LSTM) nodes have recently been used to improve state of the art in many sequential processing tasks such as speech recognition and machine translation. However, the basic structure of the LSTM node is essentially the same as when it was first conceived 25 years ago. Recently, evolutionary and reinforcement learning mechanisms have been employed to create new variations of this structure. This paper proposes a new method, evolution of a tree-based encoding of the gated memory nodes, and shows that it makes it possible to explore new variations more effectively than other methods. The method discovers nodes with multiple recurrent paths and multiple memory cells, which lead to significant improvement in the standard language modeling benchmark task. The paper also shows how the search process can be speeded up by training an LSTM network to estimate performance of candidate structures, and by encouraging exploration of novel solutions. Thus, evolutionary design of complex neural network structures promises to improve performance of deep learning architectures beyond human ability to do so.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑