arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.11034cs.CLcs.LG
收藏

条件泊松随机束搜索

Conditional Poisson Stochastic Beam Search

ETH Zürich(苏黎世联邦理工学院);Johns Hopkins University(约翰斯·霍普金斯大学);University of Cambridge(剑桥大学)

Clara Meister, Afra Amini, Tim Vieira, Ryan Cotterell

摘要:

束搜索是NLP中许多序列生成任务的默认解码策略。该算法返回的近似K-best项集是许多应用程序的分布的有用总结;然而,候选项通常表现出高度重叠,并且在我们的模型下可能会对期望值进行高度偏倚的估计。这些问题可以通过使用随机解码策略来解决。在这项工作中,我们提出了一种新的方法,把束搜索到一个随机过程:条件泊松随机束搜索。我们不是在每次迭代时取最大值集,而是根据条件泊松抽样设计对K个候选者进行抽样,而不进行替换。我们认为这是一个更自然的替代库尔等。2019年的随机束搜索(SBS)。此外,我们展示了如何在CPSBS设计下生成的样本可以用于构建一致的估计量和序列模型的样本集。在我们的实验中,我们观察到CPSBS比SBS产生更低的方差和更有效的估计,甚至在高熵设置中也有改进。

英文摘要:

Beam search is the default decoding strategy for many sequence generation tasks in NLP. The set of approximate K-best items returned by the algorithm is a useful summary of the distribution for many applications; however, the candidates typically exhibit high overlap and may give a highly biased estimate for expectations under our model. These problems can be addressed by instead using stochastic decoding strategies. In this work, we propose a new method for turning beam search into a stochastic process: Conditional Poisson stochastic beam search. Rather than taking the maximizing set at each iteration, we sample K candidates without replacement according to the conditional Poisson sampling design. We view this as a more natural alternative to Kool et. al. 2019's stochastic beam search (SBS). Furthermore, we show how samples generated under the CPSBS design can be used to build consistent estimators and sample diverse sets from sequence models. In our experiments, we observe CPSBS produces lower variance and more efficient estimators than SBS, even showing improvements in high entropy settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑