arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2112.07660cs.CL
收藏

基于格的文本生成的大规模解码

Massive-scale Decoding for Text Generation using Lattices

The University of Texas at Austin(德克萨斯大学奥斯汀分校);Amazon(亚马逊)

Jiacheng Xu, Siddhartha Reddy Jonnalagadda, Greg Durrett

摘要:

条件神经文本生成模型生成高质量的输出,但当我们真正想要的是一组多样化的选项时,通常会集中在一个模式上。我们提出了一个搜索算法来构造编码大量生成选项的格。首先,我们将解码重构为最佳优先搜索,这与束搜索不同地探索空间,并且通过避免修剪路径来提高效率。其次,我们重新审视假设重组的想法:我们可以在搜索过程中识别出相似的候选生成对,并将它们合并为近似值。无论是总结和机器翻译,我们表明,我们的算法编码成一个格成千上万的不同的选项,保持语法和高质量。该算法提供了一个基础上建立下游发电应用程序的顶部上的大规模多样化的输出。

英文摘要:

Conditional neural text generation models generate high-quality outputs, but often concentrate around a mode when what we really want is a diverse set of options. We present a search algorithm to construct lattices encoding a massive number of generation options. First, we restructure decoding as a best-first search, which explores the space differently than beam search and improves efficiency by avoiding pruning paths. Second, we revisit the idea of hypothesis recombination: we can identify pairs of similar generation candidates during search and merge them as an approximation. On both summarization and machine translation, we show that our algorithm encodes thousands of diverse options that remain grammatical and high-quality into one lattice. This algorithm provides a foundation for building downstream generation applications on top of massive-scale diverse outputs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑