自然语言生成中多样性和质量的权衡
Trading Off Diversity and Quality in Natural Language Generation
摘要：
对于开放式的语言生成任务，例如讲故事和对话，选择正确的解码算法对于控制生成质量和多样性之间的权衡至关重要。然而，目前对于哪种解码程序是最好的，甚至对于比较它们的标准，还没有达成共识。我们解决这些问题，铸造解码作为一个多目标优化问题，旨在同时最大限度地提高响应质量和多样性。我们的框架使我们能够沿着整个质量多样性频谱进行第一次大规模的解码方法评估。我们发现，当多样性是一个优先级时，所有方法的性能都相似，但当质量被视为更重要时，最近提出的核采样（Holtzman等人2019）优于所有其他评估的解码算法。我们的实验也证实了“似然陷阱”的存在，这是一种反直觉的观察，即高似然序列往往质量低得令人惊讶。我们利用我们的研究结果来创建和评估一个算法称为\emph{selective sampling}，它可跟踪近似全局归一化温度采样。
英文摘要：
For open-ended language generation tasks such as storytelling and dialogue, choosing the right decoding algorithm is critical to controlling the tradeoff between generation quality and diversity. However, there presently exists no consensus on which decoding procedure is best or even the criteria by which to compare them. We address these issues by casting decoding as a multi-objective optimization problem aiming to simultaneously maximize both response quality and diversity. Our framework enables us to perform the first large-scale evaluation of decoding methods along the entire quality-diversity spectrum. We find that when diversity is a priority, all methods perform similarly, but when quality is viewed as more important, the recently proposed nucleus sampling (Holtzman et al. 2019) outperforms all other evaluated decoding algorithms. Our experiments also confirm the existence of the `likelihood trap', the counter-intuitive observation that high likelihood sequences are often surprisingly low quality. We leverage our findings to create and evaluate an algorithm called \emph{selective sampling} which tractably approximates globally-normalized temperature sampling.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
