arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1604.00100cs.CL
收藏

语言建模的组合方法

A Compositional Approach to Language Modeling

University of Florida(佛罗里达大学)

Kushal Arora, Anand Rangarajan

摘要:

传统的语言模型把语言看作是词上概率空间上的有限状态自动机。这是一个非常强的假设,当建模的东西本身就很复杂,如语言。在本文中,我们挑战这一点,通过展示如何在以前的工作中固有的线性链假设可以转化为一个顺序组成树。然后,我们提出了一个新的模型,边缘化所有可能的组合树,从而消除任何潜在的结构假设。由于这种新模型的划分函数是棘手的,我们使用最近提出的句子级评价度量对比熵来评估我们的模型。考虑到这个新的评估指标,我们报告了超过100%的改善失真水平,超过目前最先进的基于递归神经网络的语言模型。

英文摘要:

Traditional language models treat language as a finite state automaton on a probability space over words. This is a very strong assumption when modeling something inherently complex such as language. In this paper, we challenge this by showing how the linear chain assumption inherent in previous work can be translated into a sequential composition tree. We then propose a new model that marginalizes over all possible composition trees thereby removing any underlying structural assumptions. As the partition function of this new model is intractable, we use a recently proposed sentence level evaluation metric Contrastive Entropy to evaluate our model. Given this new evaluation metric, we report more than 100% improvement across distortion levels over current state of the art recurrent neural network based language models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑