arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2203.06569cs.CL
收藏

SummaReranker:面向抽象式摘要的多任务专家混合重排序框架

SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarization

Nanyang Technological University(南洋理工大学);Salesforce Research Asia(Salesforce亚洲研究院)

Mathieu Ravaut, Shafiq Joty, Nancy F. Chen

AI总结:

本文提出多任务专家混合重排序框架 SummaReranker,对摘要候选进行二阶段选择,在 CNN-DailyMail、XSum 和 Reddit TIFU 上显著提升 PEGASUS 的 ROUGE 分数并达到新最优。

摘要:

序列到序列神经网络最近在抽象式摘要中取得了巨大成功,尤其是通过在下游数据集上微调大型预训练语言模型。这些模型通常使用束搜索解码以生成唯一的摘要。然而,搜索空间非常大,并且由于曝光偏差,这种解码并非最优。在本文中,我们展示了可以直接训练一个第二阶段模型,对一组摘要候选进行重排序。我们的专家混合模型 SummaReranker 学习选择更好的候选,并持续提升基础模型的性能。以 PEGASUS 为基础模型,我们在 CNN-DailyMail 上将 ROUGE 分数提高了 5.44%(47.16 ROUGE-1),在 XSum 上提高了 1.31%(48.12 ROUGE-1),在 Reddit TIFU 上提高了 9.34%(29.83 ROUGE-1),达到了新的最优水平。我们的代码和检查点将在 https://github.com/ntunlp/SummaReranker 上提供。

英文摘要:

Sequence-to-sequence neural networks have recently achieved great success in abstractive summarization, especially through fine-tuning large pre-trained language models on the downstream dataset. These models are typically decoded with beam search to generate a unique summary. However, the search space is very large, and with the exposure bias, such decoding is not optimal. In this paper, we show that it is possible to directly train a second-stage model performing re-ranking on a set of summary candidates. Our mixture-of-experts SummaReranker learns to select a better candidate and consistently improves the performance of the base model. With a base PEGASUS, we push ROUGE scores by 5.44% on CNN-DailyMail (47.16 ROUGE-1), 1.31% on XSum (48.12 ROUGE-1) and 9.34% on Reddit TIFU (29.83 ROUGE-1), reaching a new state-of-the-art. Our code and checkpoints will be available at https://github.com/ntunlp/SummaReranker.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑