Aya模型：一个指令微调的开放获取多语言语言模型
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
AI总结：
本文提出Aya，一个支持101种语言的大规模多语言生成式指令微调模型，覆盖超50%低资源语言，在多数任务上优于mT0和BLOOMZ且覆盖语言数翻倍，并开源了模型与指令数据集。
摘要：
大型语言模型（LLMs）的最新突破集中在少数数据丰富的语言上。如何将突破的受益范围扩大到一等公民语言之外？我们的工作引入了Aya，一个大规模多语言生成式语言模型，它能遵循101种语言的指令，其中超过50%被认为是低资源语言。Aya在大多数任务上优于mT0和BLOOMZ，同时覆盖的语言数量是它们的两倍。我们引入了广泛的新评估套件，将99种语言的多语言评估扩展到最新水平——包括判别性和生成性任务、人工评估，以及涵盖留出任务和分布内性能的模拟胜率。此外，我们对最优微调混合组成、数据修剪，以及模型的毒性、偏见和安全性进行了详细调查。我们在 https://hf.co/CohereForAI/aya-101 开源了我们的指令数据集和模型。
英文摘要：
Recent breakthroughs in large language models (LLMs) have centered around a handful of data-rich languages. What does it take to broaden access to breakthroughs beyond first-class citizen languages? Our work introduces Aya, a massively multilingual generative language model that follows instructions in 101 languages of which over 50% are considered as lower-resourced. Aya outperforms mT0 and BLOOMZ on the majority of tasks while covering double the number of languages. We introduce extensive new evaluation suites that broaden the state-of-art for multilingual eval across 99 languages -- including discriminative and generative tasks, human evaluation, and simulated win rates that cover both held-out tasks and in-distribution performance. Furthermore, we conduct detailed investigations on the optimal finetuning mixture composition, data pruning, as well as the toxicity, bias, and safety of our models. We open-source our instruction datasets and our model at https://hf.co/CohereForAI/aya-101
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
