LinguaLIFT：一种用于低资源语言推理的高效两阶段指令微调框架
LinguaLIFT: An Effective Two-stage Instruction Tuning Framework for Low-Resource Language Reasoning
AI总结：
本文提出两阶段指令微调框架 LinguaLIFT，通过语言对齐层和语码转换微调，仅用英文数据将推理能力迁移至低资源语言，并构建覆盖48种语言的 MMWP 基准，实验表现优于多个基线。
摘要：
在海量多语言预训练语料和指令微调数据的推动下，大语言模型（LLM）已展现出令人瞩目的多语言推理能力。然而，由于预训练语料中的语言不均衡，高资源与低资源语言推理任务之间存在性能差距，而现有推理基准缺乏低资源语言覆盖所导致的评估偏差又进一步加剧了这一问题。为缓解该问题，我们提出 LinguaLIFT，这是一个用于提升低资源语言推理能力的两阶段指令微调框架。LinguaLIFT 采用语言对齐层，以语码转换微调的方式捕获多语言对齐，无需多语言指令数据或平行数据，从而仅借助英文指令微调数据，将跨语言推理能力迁移到低资源语言。为全面评估多语言推理能力，我们引入 Multilingual Math World Problem（MMWP）基准，涵盖21种低资源语言、17种中资源语言和10种高资源语言。实验结果表明，LinguaLIFT 在 MMWP 及四个广泛使用的基准上均优于若干具有竞争力的基线方法。
英文摘要：
Large language models (LLMs) have exhibited impressive multilingual reasoning capabilities, driven by extensive multilingual pre-training corpora and instruction fine-tuning data. However, a performance gap exists between high- and low-resource language reasoning tasks due to the language imbalance in the pre-training corpus, which is exacerbated by evaluation bias in existing reasoning benchmarks lacking low-resource language coverage. To alleviate this issue, we propose LinguaLIFT, a two-stage instruction tuning framework for advancing low-resource language reasoning. LinguaLIFT employs a language alignment layer to capture multilingual alignment in a code-switched tuning way without requiring multilingual instruction or parallel data, thereby transferring the cross-lingual reasoning capabilities to low-resource languages through English-only instruction tuning data. To comprehensively evaluate the multilingual reasoning capabilities, we introduce the Multilingual Math World Problem (MMWP) benchmark, which spans 21 low-resource, 17 medium-resource, and 10 high-resource languages. Experimental results show that LinguaLIFT outperforms several competitive baselines across MMWP and four widely used benchmarks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
