arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.15313cs.CLcs.AIcs.LG

ArabianGPT：基于 GPT 的阿拉伯语原生大语言模型

ArabianGPT: Native Arabic GPT-based Large Language Model

Prince Sultan University（苏丹王子大学）；Robotics and Internet-of-Things Lab（机器人与物联网实验室）

Anis Koubaa, Adel Ammar, Lahouari Ghouti, Omar Najar, Serry Sibaee

阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出专为阿拉伯语设计的 ArabianGPT 系列模型及 AraNizer 分词器，通过微调在情感分析和摘要任务上显著提升性能，验证了定制化 transformer 架构对阿拉伯语 NLP 的推进作用。

摘要：

英语和基于拉丁字母的大语言模型（LLMs）的主导地位导致了阿拉伯语原生大语言模型的显著缺失。现有阿拉伯语模型中普遍包含英语 token，进一步加剧了这一差距，削弱了它们处理阿拉伯语原生复杂形态学和句法的能力。因此，无论是从理论还是实践角度，开发主要聚焦于阿拉伯语语言要素的大语言模型都具有必要性。为填补这一空白，本文提出了 ArabianGPT，这是 ArabianLLM 套件中专门为阿拉伯语设计的一系列基于 transformer 的模型。这些模型包括 ArabianGPT-0.1B 和 ArabianGPT-0.3B，在规模和复杂度上有所不同，以适配阿拉伯语细腻的语言特征。AraNizer 分词器是这些模型的关键组成部分，它针对阿拉伯文书写的独特形态学特征进行了处理，确保更准确的文本处理。在情感分析和摘要生成等任务上对模型进行微调的经验结果表明性能有显著提升。在情感分析任务中，微调后的 ArabianGPT-0.1B 模型取得了 95% 的显著准确率，相比基础模型的 56% 有大幅提高。类似地，在摘要生成任务中，微调后的模型展现出更高的 F1 分数，表明在生成简洁摘要时精确率和召回率均有改善。对微调后的 ArabianGPT 模型与其基础版本在多个基准上的对比分析揭示了性能上的细微差异，微调对问答和摘要生成等特定任务产生了积极影响。这些发现强调了微调在使 ArabianGPT 模型更紧密地契合特定 NLP 任务方面的有效性，凸显了定制化 transformer 架构在推进阿拉伯语 NLP 方面的潜力。

英文摘要：

The predominance of English and Latin-based large language models (LLMs) has led to a notable deficit in native Arabic LLMs. This discrepancy is accentuated by the prevalent inclusion of English tokens in existing Arabic models, detracting from their efficacy in processing native Arabic's intricate morphology and syntax. Consequently, there is a theoretical and practical imperative for developing LLMs predominantly focused on Arabic linguistic elements. To address this gap, this paper proposes ArabianGPT, a series of transformer-based models within the ArabianLLM suite designed explicitly for Arabic. These models, including ArabianGPT-0.1B and ArabianGPT-0.3B, vary in size and complexity, aligning with the nuanced linguistic characteristics of Arabic. The AraNizer tokenizer, integral to these models, addresses the unique morphological aspects of Arabic script, ensuring more accurate text processing. Empirical results from fine-tuning the models on tasks like sentiment analysis and summarization demonstrate significant improvements. For sentiment analysis, the fine-tuned ArabianGPT-0.1B model achieved a remarkable accuracy of 95%, a substantial increase from the base model's 56%. Similarly, in summarization tasks, fine-tuned models showed enhanced F1 scores, indicating improved precision and recall in generating concise summaries. Comparative analysis of fine-tuned ArabianGPT models against their base versions across various benchmarks reveals nuanced differences in performance, with fine-tuning positively impacting specific tasks like question answering and summarization. These findings underscore the efficacy of fine-tuning in aligning ArabianGPT models more closely with specific NLP tasks, highlighting the potential of tailored transformer architectures in advancing Arabic NLP.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑