arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.09701cs.CL
收藏

BARTpho:越南语的预训练序列到序列模型

BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamese

VinAI Research(VinAI研究院)

Nguyen Luong Tran, Duong Minh Le, Dat Quoc Nguyen

摘要:

我们提出了两个版本,BARTpho的BARTpho-syllable和BARTpho-word,这是第一个公共的大规模单语序列到序列模型预训练越南语。BARTpho使用“大型”架构和序列到序列去噪自动编码器BART的预训练方案,因此它特别适合生成式NLP任务。我们进行了实验,以比较我们的BARTpho与它的竞争对手mBART的下游任务的越南语文本摘要,并表明:在自动和人工评估,BARTpho优于强大的基线mBART,提高了国家的最先进的。我们进一步评估和比较BARTpho和mBART的越南语大写和标点符号恢复任务,也发现BARTpho是更有效的比mBART在这两个任务。我们公开发布BARTpho,以促进未来的研究和生成越南NLP任务的应用。我们的BARTpho模型可在https://github.com/VinAIResearch/BARTpho上获取

英文摘要:

We present BARTpho with two versions, BARTpho-syllable and BARTpho-word, which are the first public large-scale monolingual sequence-to-sequence models pre-trained for Vietnamese. BARTpho uses the "large" architecture and the pre-training scheme of the sequence-to-sequence denoising autoencoder BART, thus it is especially suitable for generative NLP tasks. We conduct experiments to compare our BARTpho with its competitor mBART on a downstream task of Vietnamese text summarization and show that: in both automatic and human evaluations, BARTpho outperforms the strong baseline mBART and improves the state-of-the-art. We further evaluate and compare BARTpho and mBART on the Vietnamese capitalization and punctuation restoration tasks and also find that BARTpho is more effective than mBART on these two tasks. We publicly release BARTpho to facilitate future research and applications of generative Vietnamese NLP tasks. Our BARTpho models are available at https://github.com/VinAIResearch/BARTpho

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑