arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.07596cs.LGcs.CL
收藏

面向基于偏好强化学习的生成式时间线摘要

Towards Abstractive Timeline Summarisation using Preference-based Reinforcement Learning

Intelligent Systems Laboratory, University of Bristol(布里斯托大学智能系统实验室)

Yuxuan Ye, Edwin Simpson

AI总结:

针对生成式模型在时间线摘要任务上表现不及抽取式方法的问题,提出基于偏好的强化学习方法适配预训练生成式摘要模型,在多数据集上表现更优且更契合人类偏好。

摘要:

本文提出了一种用于汇总多新闻来源报道事件时间线的新型流水线。基于Transformer的生成式摘要模型虽能生成连贯简洁的长文档摘要,但在时间线摘要(TLS)这类专项任务上,表现可能无法超越成熟的抽取式方法。抽取式摘要虽对来源更忠实,但可读性较差,且可能包含冗余或不必要信息。本文提出一种基于偏好的强化学习(PBRL)方法,用于将预训练生成式摘要模型适配到TLS任务,可克服抽取式时间线摘要的缺陷。我们定义了一个复合奖励函数,它从感兴趣关键词和成对偏好标签中学习,我们通过离线强化学习用该函数微调预训练生成式摘要模型。我们在三个数据集上开展了自动评估和人工评估,结果显示,在三个基准数据集中的两个上,本文方法优于同类抽取式TLS方法,且参与者更偏好本文方法生成的摘要,胜过抽取式TLS方法和预训练生成式模型的摘要。该方法无需昂贵的参考摘要,仅需少量偏好数据就能让生成的摘要契合人类偏好。

英文摘要:

This paper introduces a novel pipeline for summarising timelines of events reported by multiple news sources. Transformer-based models for abstractive summarisation generate coherent and concise summaries of long documents but can fail to outperform established extractive methods on specialised tasks such as timeline summarisation (TLS). While extractive summaries are more faithful to their sources, they may be less readable and contain redundant or unnecessary information. This paper proposes a preference-based reinforcement learning (PBRL) method for adapting pretrained abstractive summarisers to TLS, which can overcome the drawbacks of extractive timeline summaries. We define a compound reward function that learns from keywords of interest and pairwise preference labels, which we use to fine-tune a pretrained abstractive summariser via offline reinforcement learning. We carry out both automated and human evaluation on three datasets, finding that our method outperforms a comparable extractive TLS method on two of the three benchmark datasets, and participants prefer our method's summaries to those of both the extractive TLS method and the pretrained abstractive model. The method does not require expensive reference summaries and needs only a small number of preferences to align the generated summaries with human preferences.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑