PLANET：自回归Transformers中的动态内容规划
PLANET: Dynamic Content Planning in Autoregressive Transformers for Long-form Text Generation
摘要：
尽管预训练的语言模型在生成流畅文本方面取得了最新进展，但现有方法在需要适当的内容控制和规划以形成连贯的高级逻辑流的长格式文本生成任务中仍然存在不连贯问题。在这项工作中，我们提出了PLANET，一个新的生成框架，利用自回归自我注意力机制进行内容规划和表面实现动态。为了指导输出句子的生成，我们的框架使用潜在表示来丰富Transformer解码器，以维护基于词袋的句子级语义计划。此外，我们引入了一个新的基于连贯性的对比学习目标，以进一步提高输出的连贯性。在两个具有挑战性的长文本生成任务上进行了大量的实验，包括反驳生成和评论文章生成。自动和人工评估都表明，我们的方法显著优于强基线，并生成内容更丰富的连贯文本。
英文摘要：
Despite recent progress of pre-trained language models on generating fluent text, existing methods still suffer from incoherence problems in long-form text generation tasks that require proper content control and planning to form a coherent high-level logical flow. In this work, we propose PLANET, a novel generation framework leveraging autoregressive self-attention mechanism to conduct content planning and surface realization dynamically. To guide the generation of output sentences, our framework enriches the Transformer decoder with latent representations to maintain sentence-level semantic plans grounded by bag-of-words. Moreover, we introduce a new coherence-based contrastive learning objective to further improve the coherence of output. Extensive experiments are conducted on two challenging long-form text generation tasks including counterargument generation and opinion article generation. Both automatic and human evaluations show that our method significantly outperforms strong baselines and generates more coherent texts with richer contents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
