arXiv 2402.066192024-02-09cs.CLcs.AI

Aya数据集：面向多语言指令微调的开源集合

Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning

Cohere For AI Community（Cohere For AI 社区）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）；Cohere；Binghamton University（宾汉姆顿大学）；University of Limerick（利默里克大学）；IT University of Copenhagen（哥本哈根信息技术大学）；MIT（麻省理工学院）；University of Toronto（多伦多大学）；King Fahd University of Petroleum and Minerals（法赫德国王石油与矿物大学）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；Bloomberg LP（彭博有限合伙企业）；Cohere For AI

Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMaho… 展开作者

Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F. Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, Mike Zhang, Ramith Hettiarachchi, Joseph Wilson, Marina Machado, Luisa Souza Moura, Dominik Krzemiński, Hakimeh Fadaei, Irem Ergün, Ifeoma Okoh, Aisha Alaagib, Oshan Mudannayake, Zaid Alyafeai, Vu Minh Chien, Sebastian Ruder, Surya Guthikonda, Emad A. Alghamdi, Sebastian Gehrmann, Niklas Muennighoff, Max Bartolo, Julia Kreutzer, Ahmet Üstün, Marzieh Fadaee, Sara Hooker

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

该工作构建了覆盖65种语言的人工策划指令数据集及114种语言的5.13亿实例集合，并开源标注平台、数据集、集合与评估套件，以弥合多语言指令微调资源差距。

摘要：

数据集是现代人工智能许多突破的基础。自然语言处理（NLP）领域的许多最新成就可归因于在多样化任务集上对预训练模型进行微调，这使得大型语言模型（LLM）能够响应指令。指令微调（IFT）需要专门构建和标注的数据集。然而，现有数据集几乎都是英文的。在这项工作中，我们的主要目标是通过构建一个涵盖65种语言的人工策划的指令遵循数据集来弥合语言差距。我们与来自世界各地的流利使用者合作，收集自然的指令和完成实例。此外，我们创建了迄今为止最广泛的多语言集合，通过模板化和翻译现有数据集，涵盖114种语言，包含5.13亿个实例。总体而言，我们贡献了四项关键资源：我们开发并开源了Aya标注平台、Aya数据集、Aya集合和Aya评估套件。Aya倡议还作为参与式研究的一个宝贵案例研究，涉及来自119个国家的合作者。我们认为这是未来旨在弥合资源差距的研究合作的一个宝贵框架。

英文摘要：

Datasets are foundational to many breakthroughs in modern artificial intelligence. Many recent achievements in the space of natural language processing (NLP) can be attributed to the finetuning of pre-trained models on a diverse set of tasks that enables a large language model (LLM) to respond to instructions. Instruction fine-tuning (IFT) requires specifically constructed and annotated datasets. However, existing datasets are almost all in the English language. In this work, our primary goal is to bridge the language gap by building a human-curated instruction-following dataset spanning 65 languages. We worked with fluent speakers of languages from around the world to collect natural instances of instructions and completions. Furthermore, we create the most extensive multilingual collection to date, comprising 513 million instances through templating and translating existing datasets across 114 languages. In total, we contribute four key resources: we develop and open-source the Aya Annotation Platform, the Aya Dataset, the Aya Collection, and the Aya Evaluation Suite. The Aya initiative also serves as a valuable case study in participatory research, involving collaborators from 119 countries. We see this as a valuable framework for future research collaborations that aim to bridge gaps in resources.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。