PrahokBART:一种用于柬埔寨自然语言生成的预训练序列到序列模型
PrahokBART: A Pre-trained Sequence-to-Sequence Model for Khmer Natural Language Generation
AI总结:
PrahokBART是一种针对柬埔寨语言优化的预训练模型,通过改进语料库质量和引入语言组件,在机器翻译、文本摘要和标题生成任务中超越了现有多语言模型。
摘要:
本文介绍了PrahokBART,一种从头开始用精心挑选的柬埔寨和英语语料库训练的紧凑型预训练序列到序列模型。我们专注于提高预训练语料库的质量,并解决柬埔寨语言中被现有多语言模型忽视的语法问题,通过引入诸如词分割和规范化等语言组件。我们在三个生成任务上评估了PrahokBART:机器翻译、文本摘要和标题生成,我们的结果表明,它在性能上优于mBART50,一种强大的多语言预训练模型。此外,我们的分析提供了每个语言模块影响的见解,并评估了我们的模型在文本生成过程中处理空间的有效性,这对于柬埔寨语言文本的自然性至关重要。
英文摘要:
This work introduces {\it PrahokBART}, a compact pre-trained sequence-to-sequence model trained from scratch for Khmer using carefully curated Khmer and English corpora. We focus on improving the pre-training corpus quality and addressing the linguistic issues of Khmer, which are ignored in existing multilingual models, by incorporating linguistic components such as word segmentation and normalization. We evaluate PrahokBART on three generative tasks: machine translation, text summarization, and headline generation, where our results demonstrate that it outperforms mBART50, a strong multilingual pre-trained model. Additionally, our analysis provides insights into the impact of each linguistic module and evaluates how effectively our model handles space during text generation, which is crucial for the naturalness of texts in Khmer.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。