SaudiBERT：在沙特方言语料库上预训练的大型语言模型
SaudiBERT: A Large Language Model Pretrained on Saudi Dialect Corpora
AI总结：
提出完全基于沙特方言文本预训练的 SaudiBERT，并构建两个大规模沙特方言语料库；在11个情感分析与文本分类数据集上显著超过六种多方言阿拉伯语模型。
摘要：
本文介绍了 SaudiBERT，这是一种完全在沙特方言文本上预训练的单方言阿拉伯语语言模型。为证明该模型的有效性，我们在11个评估数据集上将 SaudiBERT 与六种不同的多方言阿拉伯语语言模型进行了比较，这些数据集分为两组：情感分析和文本分类。SaudiBERT 在这两组任务中分别取得了86.15%和87.86%的平均F1分数，显著优于所有其他对比模型。此外，我们提出了两个新的沙特方言语料库：Saudi Tweets Mega Corpus（STMC）包含超过1.41亿条沙特方言推文，Saudi Forums Corpus（SFC）则包含从五个沙特在线论坛收集的15.2 GB文本。两个语料库均用于所提出模型的预训练，并且是文献中迄今报告的最大沙特方言语料库。结果证实了 SaudiBERT 在理解和分析以沙特方言表达的阿拉伯语文本方面的有效性，在大多数任务上取得了最先进结果，并超过了研究中纳入的其他语言模型。SaudiBERT 模型已在 https://huggingface.co/faisalq/SaudiBERT 上公开发布。
英文摘要：
In this paper, we introduce SaudiBERT, a monodialect Arabic language model pretrained exclusively on Saudi dialectal text. To demonstrate the model's effectiveness, we compared SaudiBERT with six different multidialect Arabic language models across 11 evaluation datasets, which are divided into two groups: sentiment analysis and text classification. SaudiBERT achieved average F1-scores of 86.15\% and 87.86\% in these groups respectively, significantly outperforming all other comparative models. Additionally, we present two novel Saudi dialectal corpora: the Saudi Tweets Mega Corpus (STMC), which contains over 141 million tweets in Saudi dialect, and the Saudi Forums Corpus (SFC), which includes 15.2 GB of text collected from five Saudi online forums. Both corpora are used in pretraining the proposed model, and they are the largest Saudi dialectal corpora ever reported in the literature. The results confirm the effectiveness of SaudiBERT in understanding and analyzing Arabic text expressed in Saudi dialect, achieving state-of-the-art results in most tasks and surpassing other language models included in the study. SaudiBERT model is publicly available on \url{https://huggingface.co/faisalq/SaudiBERT}.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
