Jais 与 Jais-chat：以阿拉伯语为中心的基础型与指令微调开放式生成式大型语言模型
Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models
AI总结：
本文提出并开源 130 亿参数的 Jais 与 Jais-chat 阿拉伯语中心大模型，基于 GPT-3 仅解码器架构，在阿英及代码语料上训练，显著提升阿拉伯语知识与推理能力。
摘要：
我们介绍 Jais 和 Jais-chat，这是两款以阿拉伯语为中心的新型最先进开放式生成式大型语言模型（LLM），分别为基础模型和指令微调模型。这些模型基于 GPT-3 的仅解码器架构，并在阿拉伯语和英语文本（包括多种编程语言的源代码）的混合语料上进行预训练。它们拥有 130 亿个参数，基于广泛评估，其阿拉伯语知识和推理能力以显著优势优于任何现有开放的阿拉伯语及多语言模型。此外，尽管训练所用的英语数据少得多，但与规模相近的以英语为中心的开放模型相比，这些模型在英语方面仍具有竞争力。我们详细描述了模型的训练、微调、安全对齐和评估。我们发布两个开放版本：基础 Jais 模型和指令微调的 Jais-chat 变体，旨在促进阿拉伯语 LLM 研究。模型可在 https://huggingface.co/inception-mbzuai/jais-13b-chat 获取。
英文摘要：
We introduce Jais and Jais-chat, new state-of-the-art Arabic-centric foundation and instruction-tuned open generative large language models (LLMs). The models are based on the GPT-3 decoder-only architecture and are pretrained on a mixture of Arabic and English texts, including source code in various programming languages. With 13 billion parameters, they demonstrate better knowledge and reasoning capabilities in Arabic than any existing open Arabic and multilingual models by a sizable margin, based on extensive evaluation. Moreover, the models are competitive in English compared to English-centric open models of similar size, despite being trained on much less English data. We provide a detailed description of the training, the tuning, the safety alignment, and the evaluation of the models. We release two open versions of the model -- the foundation Jais model, and an instruction-tuned Jais-chat variant -- with the aim of promoting research on Arabic LLMs. Available at https://huggingface.co/inception-mbzuai/jais-13b-chat
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
