何时结束?用于神经文本生成的最佳束搜索(模束大小)
When to Finish? Optimal Beam Search for Neural Text Generation (modulo beam size)
摘要:
在诸如神经机器翻译、摘要和图像描述之类的神经文本生成中,束搜索被广泛用于提高输出文本质量。然而,在神经生成设置中,假设可以在不同的步骤中完成,这使得难以决定何时结束束搜索以确保最优性。我们提出了一种可证明的最优束搜索算法,该算法将始终返回最优分数完整假设(模束大小),并在建立最优性后立即完成(不晚于基线完成)。为了对抗神经生成对较短假设的倾向,我们还引入了一个有界长度奖励机制,该机制允许我们的束搜索算法的修改版本保持最优。神经机器翻译的实验表明,我们的原则性束搜索算法导致BLEU分数比以前提出的替代品的改善。
英文摘要:
In neural text generation such as neural machine translation, summarization, and image captioning, beam search is widely used to improve the output text quality. However, in the neural generation setting, hypotheses can finish in different steps, which makes it difficult to decide when to end beam search to ensure optimality. We propose a provably optimal beam search algorithm that will always return the optimal-score complete hypothesis (modulo beam size), and finish as soon as the optimality is established (finishing no later than the baseline). To counter neural generation's tendency for shorter hypotheses, we also introduce a bounded length reward mechanism which allows a modified version of our beam search algorithm to remain optimal. Experiments on neural machine translation demonstrate that our principled beam search algorithm leads to improvement in BLEU score over previously proposed alternatives.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。