全局引导的约束文本生成--以CommonGen为例
Constrained Text Generation with Global Guidance -- Case Study on CommonGen
摘要：
本文研究的是约束文本生成，即在一定的前提条件下生成句子。我们专注于CommonGen，基于一组概念生成文本的任务，作为受约束的文本生成的代表任务。传统的自回归解码方法主要依赖于有监督训练来最大化目标句子的似然性，但在自回归解码过程的似然性目标中，无法考虑常识和覆盖率等全局约束。在本文中，我们考虑使用强化学习来解决这个局限性，测量全局约束，包括流畅性，常识和概念覆盖率与综合得分，这作为强化学习的奖励。此外，我们还设计了一个词、片段和句子三个层次的引导解码方法。实验表明，我们的方法显着增加的概念覆盖率，并优于现有的模型在各种自动评估。
英文摘要：
This paper studies constrained text generation, which is to generate sentences under certain pre-conditions. We focus on CommonGen, the task of generating text based on a set of concepts, as a representative task of constrained text generation. Traditional methods mainly rely on supervised training to maximize the likelihood of target sentences.However, global constraints such as common sense and coverage cannot be incorporated into the likelihood objective of the autoregressive decoding process. In this paper, we consider using reinforcement learning to address the limitation, measuring global constraints including fluency, common sense and concept coverage with a comprehensive score, which serves as the reward for reinforcement learning. Besides, we design a guided decoding method at the word, fragment and sentence levels. Experiments demonstrate that our method significantly increases the concept coverage and outperforms existing models in various automatic evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
