arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2011.04640cs.CLcs.LG
收藏

标度隐马尔可夫语言模型

Scaling Hidden Markov Language Models

Cornell Tech(康奈尔科技校区)

Justin T. Chiu, Alexander M. Rush

摘要:

隐马尔可夫模型(HMM)是序列建模的基本工具,它将隐藏状态从发射结构中清晰地分离出来。然而,这种分离使得Hacker难以适应现代NLP中的大型数据集,并且由于与完全观察模型相比性能非常差,它们已经不再使用。这项工作重新审视了将Hacking扩展到语言建模数据集的挑战,从最近的神经建模方法中汲取了灵感。我们提出的方法,缩放HISTORY大规模的状态空间,同时保持高效的精确推理,紧凑的参数化,有效的正则化。实验表明,这种方法导致模型比以前的HMM和基于n-gram的方法更准确,在最先进的神经模型的性能方面取得了进展。

英文摘要:

The hidden Markov model (HMM) is a fundamental tool for sequence modeling that cleanly separates the hidden state from the emission structure. However, this separation makes it difficult to fit HMMs to large datasets in modern NLP, and they have fallen out of use due to very poor performance compared to fully observed models. This work revisits the challenge of scaling HMMs to language modeling datasets, taking ideas from recent approaches to neural modeling. We propose methods for scaling HMMs to massive state spaces while maintaining efficient exact inference, a compact parameterization, and effective regularization. Experiments show that this approach leads to models that are more accurate than previous HMM and n-gram-based methods, making progress towards the performance of state-of-the-art neural models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑