arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1811.00511cs.CL
收藏

连贯与衔接的长篇语篇生成

Towards Coherent and Cohesive Long-form Text Generation

Princeton University(普林斯顿大学);Microsoft Research AI(微软研究院人工智能)

Woon Sang Cho, Pengchuan Zhang, Yizhe Zhang, Xiujun Li, Michel Galley, Chris Brockett, Mengdi Wang, Jianfeng Gao

摘要:

生成连贯和有凝聚力的长篇文本是一项具有挑战性的任务。以前的工作依赖于大量的人类生成的文本来训练神经语言模型。然而,很少有人试图从连贯和衔接的角度明确地改善神经语言模型。在这项工作中,我们提出了一个新的神经语言模型,配备了两个神经鉴别器,提供反馈信号的水平上的句子(衔接)和段落(连贯性)。我们的模型是使用一个简单而有效的策略梯度的变体,称为负临界序列训练,这是为了消除需要训练一个单独的批评估计基线。结果表明,我们的方法的有效性,表现出强基线-基于递归注意的双向MLE训练的神经语言模型的改进。

英文摘要:

Generating coherent and cohesive long-form texts is a challenging task. Previous works relied on large amounts of human-generated texts to train neural language models. However, few attempted to explicitly improve neural language models from the perspectives of coherence and cohesion. In this work, we propose a new neural language model that is equipped with two neural discriminators which provide feedback signals at the levels of sentence (cohesion) and paragraph (coherence). Our model is trained using a simple yet efficient variant of policy gradient, called negative-critical sequence training, which is proposed to eliminate the need of training a separate critic for estimating baseline. Results demonstrate the effectiveness of our approach, showing improvements over the strong baseline -- recurrent attention-based bidirectional MLE-trained neural language model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑