arXiv 2407.021472024-07-09cs.CLcs.AI

GemmAr：通过阿拉伯语指令微调增强大语言模型

GemmAr: Enhancing LLMs Through Arabic Instruction-Tuning

Clusterlab Team（Clusterlab 团队）；Polytech INTL（国际理工学院）

Hasna Chouikhi, Manel Aloui, Cyrine Ben Hammou, Ghaith Chaabane, Haithem Kchaou, Chehir Dhaouadi

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对阿拉伯语NLP缺乏高质量指令微调数据集的问题，研究构建了多领域的InstAr-500k阿拉伯语指令数据集，微调Gemma-7B得到的GemmAr-7B-V1在多项阿拉伯语基准测试中表现优异，缩小了阿英语言模型性能差距。

摘要：

大语言模型（LLMs）已对自然语言处理（NLP）领域产生了重大影响，尤其是在英语语言方面。这些模型展现出理解和生成类人文本的能力。语言模型的成功很大程度上取决于高质量指令数据集的可用性，这类数据集包含详细的任务描述及对应的回复，对于训练模型准确应对各类提示至关重要。然而，这些资源的可用性和质量因语言而异。尽管模型在英语上表现出色，但由于缺乏用于阿拉伯语特定任务微调的数据集，它们在处理阿拉伯语等语言时往往存在困难。为解决这一问题，我们推出了InstAr-500k，这是一个全新的阿拉伯语指令数据集，通过生成和收集覆盖多个领域与指令类型的内容构建而成。我们通过在多个下游任务上微调开源Gemma-7B模型以提升其性能，以此评估该数据集。基于多项评估，我们的微调模型在多个阿拉伯语NLP基准测试上取得了优异性能。这些结果凸显了我们的数据集在提升阿拉伯语语言模型能力方面的有效性。我们的指令数据集通过提供推动阿拉伯语NLP发展的资源，缩小了英语与阿拉伯语语言模型之间的性能差距。在此基础上，我们开发了GemmAr-7B-V1模型，该模型经过专门微调，可出色完成广泛的阿拉伯语NLP任务。

英文摘要：

Large language models (LLMs) have greatly impacted the natural language processing (NLP) field, particularly for the English language. These models have demonstrated capabilities in understanding and generating human-like text. The success of language models largely depends on the availability of high-quality instruction datasets, which consist of detailed task descriptions and corresponding responses that are essential for training the models to address a variety of prompts accurately. However, the availability and quality of these resources vary by language. While models perform well in English, they often need help with languages like Arabic, due to the lack of datasets for fine-tuning Arabic-specific tasks. To address this issue, we introduce InstAr-500k, a new Arabic instruction dataset created by generating and collecting content that covers several domains and instruction types. We assess this dataset by fine-tuning an open-source Gemma-7B model on several downstream tasks to improve its functionality. Based on multiple evaluations, our fine-tuned model achieves excellent performance on several Arabic NLP benchmarks. These outcomes emphasize the effectiveness of our dataset in elevating the capabilities of language models for Arabic. Our instruction dataset bridges the performance gap between English and Arabic language models by providing resources that amplify Arabic NLP development. Building on this foundation, we developed a model, GemmAr-7B-V1, specifically tuned to excel at a wide range of Arabic NLP tasks.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文