使用图形表示导航以实现神经语言模型的快速和可扩展解码
Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models
摘要:
神经语言模型(NLM)最近获得了新的兴趣,在许多自然语言处理(NLP)任务实现了最先进的性能。然而,NLM在计算上要求很高,主要是由于softmax层在大词汇表上的计算成本。我们观察到,在许多NLP任务的解码中,仅需要精确计算前K个假设的概率,并且K通常远小于词汇量。本文提出了一种新的softmax层近似算法,称为快速图解码器(FGD),它快速识别,对于给定的上下文,一组K字是最有可能出现根据NLM。我们证明了FGD将解码时间减少了一个数量级,同时在神经机器翻译和语言建模任务上达到了接近完整softmax基线的准确性。我们还证明了softmax逼近质量的理论保证。
英文摘要:
Neural language models (NLMs) have recently gained a renewed interest by achieving state-of-the-art performance across many natural language processing (NLP) tasks. However, NLMs are very computationally demanding largely due to the computational cost of the softmax layer over a large vocabulary. We observe that, in decoding of many NLP tasks, only the probabilities of the top-K hypotheses need to be calculated preciously and K is often much smaller than the vocabulary size. This paper proposes a novel softmax layer approximation algorithm, called Fast Graph Decoder (FGD), which quickly identifies, for a given context, a set of K words that are most likely to occur according to a NLM. We demonstrate that FGD reduces the decoding time by an order of magnitude while attaining close to the full softmax baseline accuracy on neural machine translation and language modeling tasks. We also prove the theoretical guarantee on the softmax approximation quality.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。