MURI：通过反向指令为低资源语言构建高质量指令微调数据集
MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructions
AI总结：
本文提出MURI方法，利用反向指令和翻译流水线，无需人工标注即可为200种低资源语言生成超200万指令-输出对，经评估和微调实验验证，有效提升NLU和开放式生成性能。
摘要：
指令微调通过使大型语言模型（LLMs）与跨多样任务的人类偏好对齐来增强其性能。传统创建指令微调数据集的方法因依赖数据标注，在低资源语言上面临严重挑战。本研究提出了一种新颖方法——多语言反向指令（MURI），该方法无需人工标注者或预先存在的多语言模型，即可为低资源语言生成高质量的指令微调数据集。利用反向指令和翻译流水线，MURI从低资源语言中已有的由人类撰写的文本中生成指令-输出对。该方法通过从不同本土领域获取文本并应用过滤器排除不当内容，确保了文化相关性和多样性。我们的数据集MURI-IT包含超过200种语言的200多万个指令-输出对。母语者的评估和mT5模型的微调实验证明了该方法在自然语言理解（NLU）和开放式生成方面的有效性。我们在https://github.com/akoksal/muri公开发布数据集和模型。
英文摘要：
Instruction tuning enhances large language models (LLMs) by aligning them with human preferences across diverse tasks. Traditional approaches to create instruction tuning datasets face serious challenges for low-resource languages due to their dependence on data annotation. This work introduces a novel method, Multilingual Reverse Instructions (MURI), which generates high-quality instruction tuning datasets for low-resource languages without requiring human annotators or pre-existing multilingual models. Utilizing reverse instructions and a translation pipeline, MURI produces instruction-output pairs from existing human-written texts in low-resource languages. This method ensures cultural relevance and diversity by sourcing texts from different native domains and applying filters to eliminate inappropriate content. Our dataset, MURI-IT, includes more than 2 million instruction-output pairs across 200 languages. Evaluation by native speakers and fine-tuning experiments with mT5 models demonstrate the approach's effectiveness for both NLU and open-ended generation. We publicly release datasets and models at https://github.com/akoksal/muri.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
