TaCo：通过翻译辅助的思维链过程增强大语言模型中低资源语言的跨语言迁移
TaCo: Enhancing Cross-Lingual Transfer for Low-Resource Languages in LLMs through Translation-Assisted Chain-of-Thought Processes
AI总结：
本文提出含132种语言翻译的MITS数据集及翻译辅助思维链方法TaCo，通过课程学习低成本增强LLM低资源跨语言迁移，低资源语言评测性能较单纯指令微调翻倍。
摘要：
构建多语言大语言模型（LLM）面临重大挑战。预训练或微调LLM以使其采用新语言显然成本极高。此外，在多语言场景中，基准数据集以及用于衡量模型性能的指标也存在局限性。本文针对上述两个挑战提出了具有成本效益的解决方案。首先，我们介绍了多语言指令微调数据集（Multilingual Instruction-Tuning Dataset，MITS），该数据集由Alpaca-52K、Dolly-15K和Vicuna Benchmark翻译成132种语言组成。其次，我们提出了一种名为TaCo: Translation-Assisted Cross-Linguality（翻译辅助跨语言性）的新方法，该方法在思维链过程中利用翻译，通过课程学习过程对LLM进行新语言的指令微调。作为概念验证，我们对经过指令微调的Guanaco-33B模型进行了实验，使用我们提出的TaCo方法在三种低资源语言和一种高资源语言上进一步执行指令微调。结果表明，TaCo方法在Vicuna Benchmark数据集中针对一种低资源语言获得了令GPT-4印象深刻的82%分数，与仅进行指令微调相比性能翻倍。此外，TaCo在构建多语言LLM方面展现出前景，即使对低资源语言也是如此。我们已经发布了数据集和模型适配器（https://github.com/UNHSAILLab/TaCo），鼓励研究社区利用这些资源推进多语言LLM方面的工作。
英文摘要：
Creating multilingual LLMs poses a significant challenge. Pretraining or fine-tuning LLMs to adopt new languages is evidently very costly. Furthermore, there exist limitations concerning benchmark datasets and the metrics used to measure model performance in multilingual settings. This paper proposes cost-effective solutions to both aforementioned challenges. Firstly, we introduce the Multilingual Instruction-Tuning Dataset (MITS), comprised of Alpaca-52K, Dolly-15K, and Vicuna Benchmark translations into 132 languages. Secondly, we propose a new method called \emph{TaCo: Translation-Assisted Cross-Linguality}, which utilizes translations in a chain-of-thought process to instruction-tune LLMs on new languages through a curriculum-learning process. As a proof of concept, we experimented with the instruction-tuned Guanaco-33B model, performing further instruction tuning using our proposed TaCo method in three low-resource languages and one high-resource language. Our results indicate that the TaCo method impresses GPT-4 with an 82\% score for a low-resource language in the Vicuna Benchmark dataset, doubling the performance in contrast to instruction tuning alone. Furthermore, TaCo shows promise in creating multilingual LLMs, even for low-resource languages. We have released our datasets and model adapters\footnote{https://github.com/UNHSAILLab/TaCo} , encouraging the research community to utilize these resources to advance work on multilingual LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
