arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2103.05070cs.CLcs.AIcs.LG
收藏

通过标记简化文本

Text Simplification by Tagging

Grammarly(Grammarly公司)

Kostiantyn Omelianchuk, Vipul Raheja, Oleksandr Skurzhanskyi

摘要:

基于编辑的方法最近在多个单语序列转导任务上显示出有希望的结果。与传统的序列到序列(Seq2Seq)模型相比,这些方法在并行语料库上训练时从头开始学习生成文本,这些方法已被证明更有效,因为它们能够学习进行快速准确的转换,同时利用强大的预训练语言模型。受这些想法的启发,我们提出了TST,一个简单而高效的文本简化系统,基于序列标记,利用预先训练的基于transformer的编码器。我们的系统在现有系统的训练和推理中进行了简单的数据增强和调整,这使得它不太依赖于大量并行训练数据,对输出提供了更多的控制,并实现了更快的推理速度。我们最好的模型在该任务的基准测试数据集上实现了接近最先进的性能。由于它是完全非自回归的,因此它的推理速度比当前最先进的文本简化系统快11倍以上。

英文摘要:

Edit-based approaches have recently shown promising results on multiple monolingual sequence transduction tasks. In contrast to conventional sequence-to-sequence (Seq2Seq) models, which learn to generate text from scratch as they are trained on parallel corpora, these methods have proven to be much more effective since they are able to learn to make fast and accurate transformations while leveraging powerful pre-trained language models. Inspired by these ideas, we present TST, a simple and efficient Text Simplification system based on sequence Tagging, leveraging pre-trained Transformer-based encoders. Our system makes simplistic data augmentations and tweaks in training and inference on a pre-existing system, which makes it less reliant on large amounts of parallel training data, provides more control over the outputs and enables faster inference speeds. Our best model achieves near state-of-the-art performance on benchmark test datasets for the task. Since it is fully non-autoregressive, it achieves faster inference speeds by over 11 times than the current state-of-the-art text simplification system.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑