arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2105.13635cs.CL
收藏

文本摘要的噪声一致性训练

Noised Consistency Training for Text Summarization

Junnan Liu, Qianren Mao, Bang Liu, Hao Peng, Hongdong Zhu, Jianxin Li

摘要:

神经抽象摘要方法通常需要大量带标签的训练数据。然而,标记大量的摘要数据往往是禁止的,由于时间,财务和专业知识的限制,这限制了摘要系统的实用性,以实际应用。在本文中,我们认为,这种限制可以克服半监督的方法:一致性训练,这是利用大量的未标记的数据,以提高性能的监督学习在一个小的语料库。一致性正则化半监督学习可以将模型预测正则化为对应用于输入文章的小噪声不变。通过添加噪声未标记语料库来帮助正则化一致性训练,该框架在不使用完整数据集的情况下获得了比较性能。特别是,我们已经验证了利用大量未标记的数据可以体面地提高监督学习在标记数据集不足的情况下的性能。

英文摘要:

Neural abstractive summarization methods often require large quantities of labeled training data. However, labeling large amounts of summarization data is often prohibitive due to time, financial, and expertise constraints, which has limited the usefulness of summarization systems to practical applications. In this paper, we argue that this limitation can be overcome by a semi-supervised approach: consistency training which is to leverage large amounts of unlabeled data to improve the performance of supervised learning over a small corpus. The consistency regularization semi-supervised learning can regularize model predictions to be invariant to small noise applied to input articles. By adding noised unlabeled corpus to help regularize consistency training, this framework obtains comparative performance without using the full dataset. In particular, we have verified that leveraging large amounts of unlabeled data decently improves the performance of supervised learning over an insufficient labeled dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑