arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1707.06130cs.CL
收藏

使用密集连接循环神经网络改进语言建模

Improving Language Modeling using Densely Connected Recurrent Neural Networks

Ghent University(根特大学);imec(比利时微电子研究中心)

Fréderic Godin, Joni Dambre, Wesley De Neve

摘要:

在本文中,我们将稠密连接层的新概念引入循环神经网络。我们评估我们提出的架构上的Penn Treebank语言建模任务。我们表明,与使用dropout训练的标准堆叠2层LSTM模型相比,我们可以用少六倍的参数获得类似的困惑分数(Zaremba et al. 2014)。与目前使用的跳过连接,我们表明,密集连接只有几个堆栈层与跳过连接已经产生显着的困惑减少。

英文摘要:

In this paper, we introduce the novel concept of densely connected layers into recurrent neural networks. We evaluate our proposed architecture on the Penn Treebank language modeling task. We show that we can obtain similar perplexity scores with six times fewer parameters compared to a standard stacked 2-layer LSTM model trained with dropout (Zaremba et al. 2014). In contrast with the current usage of skip connections, we show that densely connecting only a few stacked layers with skip connections already yields significant perplexity reductions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑