Directed Beam Search：即插即用的词法约束语言生成
Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation
摘要：
大型预训练语言模型能够生成逼真的文本。然而，控制这些模型使得生成的文本满足词汇约束，即，包含特定的单词，是一个具有挑战性的问题。考虑到最先进的语言模型太大而无法在可管理的时间内从头开始训练，因此希望在不重新训练它们的情况下控制这些模型。能够做到这一点的方法被称为即插即用。最近的即插即用方法已经成功地在具有受限搜索空间的任务中约束小的双向语言模型以及前向模型，机器翻译。然而，控制大型基于transformer的模型以满足词汇约束而不重新训练它们仍然是一个挑战。在这项工作中，我们提出了定向束搜索（DBS），即插即用的词汇约束语言生成的方法。该方法适用于任何语言模型，易于实现，可用于一般的语言生成。在我们的实验中，我们使用DBS来控制GPT-2。我们展示了它的性能上的关键字到短语的生成，我们获得了可比的结果作为一个国家的最先进的非即插即用模型的词汇约束的故事生成。
英文摘要：
Large pre-trained language models are capable of generating realistic text. However, controlling these models so that the generated text satisfies lexical constraints, i.e., contains specific words, is a challenging problem. Given that state-of-the-art language models are too large to be trained from scratch in a manageable time, it is desirable to control these models without re-training them. Methods capable of doing this are called plug-and-play. Recent plug-and-play methods have been successful in constraining small bidirectional language models as well as forward models in tasks with a restricted search space, e.g., machine translation. However, controlling large transformer-based models to meet lexical constraints without re-training them remains a challenge. In this work, we propose Directed Beam Search (DBS), a plug-and-play method for lexically constrained language generation. Our method can be applied to any language model, is easy to implement and can be used for general language generation. In our experiments we use DBS to control GPT-2. We demonstrate its performance on keyword-to-phrase generation and we obtain comparable results as a state-of-the-art non-plug-and-play model for lexically constrained story generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
