arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1803.08240cs.CLcs.AIcs.NE
收藏

多尺度神经语言建模分析

An Analysis of Neural Language Modeling at Multiple Scales

Salesforce Research(Salesforce研究院)

Stephen Merity, Nitish Shirish Keskar, Richard Socher

摘要:

语言建模中的许多主要方法都引入了新颖、复杂和专门的架构。我们采用现有的基于LSTM和QRNN的最先进的单词级语言模型,并将其扩展到更大的词汇表和字符级粒度。当适当调整时,LSTM和QRNN分别在字符级(Penn Treebank,enwik 8)和单词级(WikiText-103)数据集上实现了最先进的结果。使用单个现代GPU,只需12小时(WikiText-103)到2天(enwik 8)即可获得结果。

英文摘要:

Many of the leading approaches in language modeling introduce novel, complex and specialized architectures. We take existing state-of-the-art word level language models based on LSTMs and QRNNs and extend them to both larger vocabularies as well as character-level granularity. When properly tuned, LSTMs and QRNNs achieve state-of-the-art results on character-level (Penn Treebank, enwik8) and word-level (WikiText-103) datasets, respectively. Results are obtained in only 12 hours (WikiText-103) to 2 days (enwik8) using a single modern GPU.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑