DPRK-BERT：最高的语言模型
DPRK-BERT: The Supreme Language Model
摘要：
深度语言模型在NLP领域取得了显著的成功。训练深度语言模型的标准方法是在大型未标记语料库上从头开始采用无监督学习。然而，这种大型语料库仅适用于广泛采用和高资源的语言和领域。本研究提出了朝鲜语的第一个深层语言模型DPRK-BERT。我们实现这一点，通过编译朝鲜语的第一个未标记的语料库和微调预先存在的韩国语言模型。我们将所提出的模型与现有的方法进行比较，并在两个朝鲜数据集上显示出显着的改进。我们还提出了一个跨语言版本的这个模型，产生更好的泛化在两种韩语。最后，我们提供了与朝鲜语相关的各种NLP工具，以促进未来的研究。
英文摘要：
Deep language models have achieved remarkable success in the NLP domain. The standard way to train a deep language model is to employ unsupervised learning from scratch on a large unlabeled corpus. However, such large corpora are only available for widely-adopted and high-resource languages and domains. This study presents the first deep language model, DPRK-BERT, for the DPRK language. We achieve this by compiling the first unlabeled corpus for the DPRK language and fine-tuning a preexisting the ROK language model. We compare the proposed model with existing approaches and show significant improvements on two DPRK datasets. We also present a cross-lingual version of this model which yields better generalization across the two Korean languages. Finally, we provide various NLP tools related to the DPRK language that would foster future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
