对话生成中的驯服重复
Taming Repetition in Dialogue Generation
摘要:
预训练语言模型的浪潮一直在不断提高机器生成的对话的质量,然而,一些生成的响应仍然遭受过度重复,有时重复话语中的单词,有时重复自我生成的响应中的单词,或两者兼而有之。不适当的单词重复会显著降低生成文本的质量。惩罚抽样是一种流行的解决方案,它降低了推理过程中现有词的抽样概率,但它非常容易受到静态权重设置不当的影响。设置太高会产生奇怪和不切实际的句子,而设置太低则会使抑制重复的任务变得微不足道。为了弥补上述方法的缺点,我们设计了一个上下文感知分类器来显式决定何时允许重复以及何时采用惩罚抽样。这样的分类器可以很容易地与现有的解码方法集成,在适当的情况下减少重复,同时保持文本的多样性。实验结果表明,该方法可以生成更高质量和更真实的对话。
英文摘要:
The wave of pre-training language models has been continuously improving the quality of the machine-generated conversations, however, some of the generated responses still suffer from excessive repetition, sometimes repeating words from utterance, sometimes repeating words within self-generated responses, or both. Inappropriate repetition of words can significantly degrade the quality of the generated texts. Penalized sampling is one popular solution, reducing the sampling probability of existing words during inference, however, it is highly vulnerable to the inappropriate setting of the static weight. Setting it too high can yield strange and unrealistic sentences while setting it too low makes the task of suppressing repetition trivial. To remedy the shortcomings of the above methods, we design a context-aware classifier to explicitly decide when to allow repetition and when to employ penalized sampling. Such a classifier can be easily integrated with existing decoding methods, reducing repetitions where appropriate while preserving the diversity of the text. Experimental results demonstrate that our method can generate higher quality and more authentic dialogues.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。