打破多语言数学推理中的语言壁垒：洞察与观察
Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations
AI总结：
本文构建了首个涵盖十种语言的多语言数学推理指令数据集MGSM8KInstruct，并提出训练策略构建MathOctopus模型，在少样本场景下超越ChatGPT，同时揭示了多语言语料库对提升单语言性能的积极作用。
摘要：
现有研究主要集中在开发用于单语言数学推理的强大语言学习模型（LLMs），鲜有探索在多语言语境中保持其有效性。为弥补这一空白，本文率先探索并训练了强大的多语言数学推理（xMR）LLMs。首先，通过利用翻译，我们构建了首个多语言数学推理指令数据集MGSM8KInstruct，涵盖十种不同语言，从而解决了xMR任务中训练数据稀缺的问题。基于收集的数据集，我们提出了不同的训练策略来构建强大的xMR LLMs，命名为MathOctopus，其显著优于传统开源LLMs，并在少样本场景中展现出超越ChatGPT的优越性。值得注意的是，MathOctopus-13B达到47.6%的准确率，在MGSM测试集上超过了ChatGPT的46.3%。除了显著的结果外，我们从大量实验中发掘了若干关键观察和洞察：（1）将拒绝采样策略扩展到多语言语境时，对模型性能有效，尽管效果有限。（2）在多种语言中使用平行语料库进行数学监督微调（SFT），不仅显著提升了模型的多语言性能，还提高了其单语言性能。这表明，构建多语言语料库可被视为增强模型在特定语言（尤其是数学推理任务）中性能的重要策略。例如，MathOctopus-7B在GSM8K测试集上将其在英语上训练的对应模型的性能从42.2%提升至50.8%。代码可在https://github.com/microsoft/MathOctopus获取。
英文摘要：
Existing research predominantly focuses on developing powerful language learning models (LLMs) for mathematical reasoning within monolingual languages, with few explorations in preserving efficacy in a multilingual context. To bridge this gap, this paper pioneers exploring and training powerful Multilingual Math Reasoning (xMR) LLMs. Firstly, by utilizing translation, we construct the first multilingual math reasoning instruction dataset, MGSM8KInstruct, encompassing ten distinct languages, thus addressing the issue of training data scarcity in xMR tasks. Based on the collected dataset, we propose different training strategies to build powerful xMR LLMs, named MathOctopus, notably outperform conventional open-source LLMs and exhibit superiority over ChatGPT in few-shot scenarios. Notably, MathOctopus-13B reaches 47.6% accuracy which exceeds ChatGPT 46.3% on MGSM testset. Beyond remarkable results, we unearth several pivotal observations and insights from extensive experiments: (1) When extending the rejection sampling strategy to the multilingual context, it proves effective for model performances, albeit limited. (2) Employing parallel corpora for math Supervised Fine-Tuning (SFT) across multiple languages not only significantly enhances model performance multilingually but also elevates their monolingual performance. This indicates that crafting multilingual corpora can be regarded as a vital strategy for enhancing model performance in a specific language, especially in mathematical reasoning tasks. For instance, MathOctopus-7B improves its counterparts that trained on English from 42.2% to 50.8% on GSM8K testset. Codes are available at https://github.com/microsoft/MathOctopus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
