用于序列建模的乘性LSTM
Multiplicative LSTM for sequence modelling
摘要：
我们介绍了乘法LSTM（mLSTM），这是一种用于序列建模的递归神经网络架构，它结合了长短期记忆（LSTM）和乘法递归神经网络架构。mLSTM的特点是它能够为每个可能的输入提供不同的递归转换函数，我们认为这使得它更适合自回归密度估计。我们凭经验证明，mLSTM在一系列字符级语言建模任务中优于标准LSTM及其深度变体。在这个版本的论文中，我们正则化mLSTM，以实现text 8上的1.27 bits/char和Hutter Prize上的1.24 bits/char。我们还在WikiText-2数据集上应用了一个纯字节级的mLSTM，以实现1.26 bits/char的字符级熵，对应于88.8的单词级困惑度，这与在同一任务中以类似方式正则化的单词级LSTM相当。
英文摘要：
We introduce multiplicative LSTM (mLSTM), a recurrent neural network architecture for sequence modelling that combines the long short-term memory (LSTM) and multiplicative recurrent neural network architectures. mLSTM is characterised by its ability to have different recurrent transition functions for each possible input, which we argue makes it more expressive for autoregressive density estimation. We demonstrate empirically that mLSTM outperforms standard LSTM and its deep variants for a range of character level language modelling tasks. In this version of the paper, we regularise mLSTM to achieve 1.27 bits/char on text8 and 1.24 bits/char on Hutter Prize. We also apply a purely byte-level mLSTM on the WikiText-2 dataset to achieve a character level entropy of 1.26 bits/char, corresponding to a word level perplexity of 88.8, which is comparable to word level LSTMs regularised in similar ways on the same task.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
