arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.07074cs.CL
收藏

摘要、大纲和阐述:通过分层监督从摘要中生成长文本

Summarize, Outline, and Elaborate: Long-Text Generation via Hierarchical Supervision from Extractive Summaries

Zhejiang University(浙江大学);Peking University(北京大学);Peng Cheng Laboratory(鹏城实验室)

Xiaofei Sun, Zijun Sun, Yuxian Meng, Jiwei Li, Chun Fan

摘要:

生成连贯的长文本的困难在于,现有的模型绝大多数集中在预测本地单词,而不能对生成什么或捕获文本块之间的高层次话语依赖关系做出高层次的计划。受人类写作过程的启发,首先列出要点列表或目录,然后扩展每个要点以形成整篇文章,我们提出{ \it SOE},这是一个管道系统,涉及总结,概述和详细说明长文本生成:该模型首先概述了长文本的不同部分的摘要,然后详细说明每个要点以生成相应的部分。为了避免劳动密集型的摘要征求过程中,我们提出了{ \it重建}策略,它提取段摘要在无监督的方式,通过选择其最丰富的部分,以重建段。该生成系统具有以下优点:(1)摘要为文本生成提供了高层次的指导,避免了单个单词预测的局部最小值;(2)在摘要之间的条件依赖性中捕获高级话语依赖性并且在摘要扩展过程期间保留高级话语依赖性,以及(3)另外,通过将上下文表示为简明的摘要,我们能够考虑更多的上下文。大量的实验表明,SOE产生的长文本具有更好的质量,以及更快的收敛速度。

英文摘要:

The difficulty of generating coherent long texts lies in the fact that existing models overwhelmingly focus on predicting local words, and cannot make high level plans on what to generate or capture the high-level discourse dependencies between chunks of texts. Inspired by human writing processes, where a list of bullet points or a catalog is first outlined, and then each bullet point is expanded to form the whole article, we propose {\it SOE}, a pipelined system that involves of summarizing, outlining and elaborating for long text generation: the model first outlines the summaries for different segments of long texts, and then elaborates on each bullet point to generate the corresponding segment. To avoid the labor-intensive process of summary soliciting, we propose the {\it reconstruction} strategy, which extracts segment summaries in an unsupervised manner by selecting its most informative part to reconstruct the segment. The proposed generation system comes with the following merits: (1) the summary provides high-level guidance for text generation and avoids the local minimum of individual word predictions; (2) the high-level discourse dependencies are captured in the conditional dependencies between summaries and are preserved during the summary expansion process and (3) additionally, we are able to consider significantly more contexts by representing contexts as concise summaries. Extensive experiments demonstrate that SOE produces long texts with significantly better quality, along with faster convergence speed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑