Bactrian-X：采用低秩适配的多语言可复现指令跟随模型
Bactrian-X: Multilingual Replicable Instruction-Following Models with Low-Rank Adaptation
AI总结：
针对多语言指令微调数据集匮乏的问题，研究团队构建了涵盖52种语言、340万条对的Bactrian-X多语言平行数据集，用LoRA训练轻量适配器，实验证明其性能优于原始及现有指令微调模型，相关资源已开源。
摘要：
指令微调在提升大语言模型性能方面展现出巨大潜力。然而，由于缺乏跨不同语言的高质量指令-响应数据集，多语言指令微调的研究受到了限制。为填补这一空白，我们提出Bactrian-X，这是一个涵盖52种语言、包含340万条指令-响应对的综合性多语言平行数据集。利用该数据集，我们采用低秩适配（LoRA）训练了一组适配器，这些适配器是可与大语言模型无缝集成的轻量组件，参数量远低于基座模型，因此易于替换，可作为不同语言或语言组的插件使用。在多种多语言评估设置下开展的大量实验表明，基于Bactrian-X进行LoRA训练得到的模型，性能优于原始模型和现有指令微调模型。代码与模型已公开发布于https://github.com/mbzuai-nlp/bactrian-x。
英文摘要：
Instruction tuning has shown great promise in improving the performance of large language models. However, research on multilingual instruction tuning has been limited due to the scarcity of high-quality instruction-response datasets across different languages. To bridge this gap, we present Bactrian-X, a comprehensive multilingual parallel dataset of 3.4 million instruction-response pairs across 52 languages. Leveraging this dataset, we train a set of adapters using low-rank adaptation (LoRA), which are lightweight components that seamlessly integrate with large language models. These adapters have a substantially lower parameter count than the base model, making them easily replaceable and usable as plug-ins for different languages or language groups. Extensive experiments in various multilingual evaluation settings demonstrate that models derived from LoRA-based training over Bactrian-X outperform both the vanilla models and existing instruction-tuned models. The code and models are publicly available at https://github.com/mbzuai-nlp/bactrian-x
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
