arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1811.09353cs.CL
收藏

使用分段神经语言模型学习发现、基础和使用单词

Learning to Discover, Ground and Use Words with Segmental Neural Language Models

DeepMind(深度思维);University of Oxford(牛津大学)

Kazuya Kawakami, Chris Dyer, Phil Blunsom

摘要:

我们提出了一个分段神经语言模型,结合了神经网络的泛化能力与发现未分段字符序列中潜在的单词样单元的能力。与以前将分词视为一项孤立任务的分割模型相比,我们的模型统一了单词发现,学习单词如何组合在一起形成句子,以及通过调节视觉上下文模型,单词的含义如何在非语言模态的表示中扎根。实验表明,无条件模型比字符LSTM模型更好地学习预测分布,与非参数贝叶斯分词模型竞争性地发现单词,并且以视觉上下文为条件的语言建模提高了两者的性能。

英文摘要:

We propose a segmental neural language model that combines the generalization power of neural networks with the ability to discover word-like units that are latent in unsegmented character sequences. In contrast to previous segmentation models that treat word segmentation as an isolated task, our model unifies word discovery, learning how words fit together to form sentences, and, by conditioning the model on visual context, how words' meanings ground in representations of non-linguistic modalities. Experiments show that the unconditional model learns predictive distributions better than character LSTM models, discovers words competitively with nonparametric Bayesian word segmentation models, and that modeling language conditional on visual context improves performance on both.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑