关于将正则语言高效表示为RNN的研究
On Efficiently Representing Regular Languages as RNNs
AI总结：
本研究推广了Hewitt等人的构建方法，证明RNN能高效表示的语言模型类别更广，即可由带特定栈更新函数的有界栈下推自动机表示的类别，为RNN的归纳偏置提供了新解释。
摘要：
Hewitt等人（2020）的近期研究对循环神经网络（RNN）作为语言模型（LMs）的实证成功给出了一种解释，证明RNN能够高效表示人类语言中普遍存在的有界层次结构，这表明RNN的成功可能与其建模层次结构的能力相关。然而，仔细审视Hewitt等人（2020）的构建方式可以发现，它并非天然局限于层次结构。这就引出了一个自然的问题：RNN还能高效表示哪些其他类别的语言模型？为此，我们推广了Hewitt等人（2020）的构建方法，证明RNN能够高效表示的语言模型类别比此前宣称的更大——具体而言，是那些可由带有有界栈和特定栈更新函数的下推自动机表示的语言模型。总体而言，用RNN语言模型表示这一多样化类别语言模型的效率，为理解其归纳偏置提供了新的解释。
英文摘要：
Recent work by Hewitt et al. (2020) provides an interpretation of the empirical success of recurrent neural networks (RNNs) as language models (LMs). It shows that RNNs can efficiently represent bounded hierarchical structures that are prevalent in human language. This suggests that RNNs' success might be linked to their ability to model hierarchy. However, a closer inspection of Hewitt et al.'s (2020) construction shows that it is not inherently limited to hierarchical structures. This poses a natural question: What other classes of LMs can RNNs efficiently represent? To this end, we generalize Hewitt et al.'s (2020) construction and show that RNNs can efficiently represent a larger class of LMs than previously claimed -- specifically, those that can be represented by a pushdown automaton with a bounded stack and a specific stack update function. Altogether, the efficiency of representing this diverse class of LMs with RNN LMs suggests novel interpretations of their inductive bias.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
