arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2004.13838cs.CL
收藏

循环神经网络有多混乱?

How Chaotic Are Recurrent Neural Networks?

University of Alberta(阿尔伯塔大学);Alberta Machine Intelligence Institute (Amii)(阿尔伯塔机器智能研究所(Amii))

Pourya Vakilipourtakalou, Lili Mou

摘要:

循环神经网络(RNN)是一种非线性动态系统。以前的工作认为,RNN可能会受到混沌现象的影响,系统对初始状态敏感,从长远来看是不可预测的。然而,在本文中,我们进行了系统的实证分析,表明香草或长短期记忆(LSTM)RNN在实际应用(如文本生成)的训练过程中不会表现出混沌行为。我们的研究结果表明,未来在这个方向上的工作应该解决RNN非线性动力学的另一面。

英文摘要:

Recurrent neural networks (RNNs) are non-linear dynamic systems. Previous work believes that RNN may suffer from the phenomenon of chaos, where the system is sensitive to initial states and unpredictable in the long run. In this paper, however, we perform a systematic empirical analysis, showing that a vanilla or long short term memory (LSTM) RNN does not exhibit chaotic behavior along the training process in real applications such as text generation. Our findings suggest that future work in this direction should address the other side of non-linear dynamics for RNN.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑