通过层交换重新思考多语言推理差距
Rethinking the Multilingual Reasoning Gap with Layer Swap
AI总结：
本文通过构建多语言推理数据集并微调专家模型，发现本地推理与英语枢轴推理的性能差距远小于先前报道，并提出层交换方法将英语专家的推理中间层迁移到本地专家，以缩小差距。
摘要：
最近的推理大语言模型在生成思维链（CoT）时主要使用英语，即使提示使用非英语语言。先前的研究表明，强制CoT保持输入语言（本地推理）会显著降低性能，而允许模型用英语推理然后用输入语言回答（英语枢轴推理）则表现更好。然而，大多数关于这种本地推理差距的研究依赖于推理时的干预或有限的本地语言训练数据。我们在更大规模且可比监督下重新审视这一比较。我们构建了涵盖六种语言（英语、法语、德语、西班牙语、中文和斯瓦希里语）的长篇多语言推理数据集；在Qwen/Qwen3-8B-Base基础上微调本地和英语枢轴两种模式的专家模型，并在数学、科学、通用知识和代码任务上进行评估。在此设置下，五种非英语语言的平均本地推理差距缩小至1.9-3.5%，远小于先前报道。对本地专家的权重空间分析显示，中间层的微调更新具有对齐性，而外层则存在分歧。这表明存在一个很大程度上与语言无关的推理核心，周围环绕着特定语言的层。利用这一结构，我们引入了层交换方法：将英语专家更强的推理中间层迁移到每个本地专家中，从而在保留目标语言CoT的同时，几乎消除了五种非英语语言的本地推理差距。我们发布了所有模型和数据集。
英文摘要：
Recent reasoning Large Language Models produce a chain-of-thought (CoT) predominantly in English, even when prompted in non-English languages. Prior work suggests that forcing the CoT to remain in the input language (native reasoning) substantially degrades performance relative to allowing the model to reason in English before answering in the input language (English-pivoted reasoning). However, most studies of this native reasoning gap rely on inference-time interventions or limited native-language training data. We revisit this comparison at a larger scale and under comparable supervision. We construct long multilingual reasoning datasets across six languages (English, French, German, Spanish, Chinese and Swahili); fine-tune specialists in both native and English-pivoted regimes on top of Qwen/Qwen3-8B-Base, and evaluate across mathematics, science, general knowledge, and code. In this setting, the average native reasoning gap shrinks to 1.9--3.5% across the five non-English languages, considerably smaller than previously reported. Weight-space analysis of the native specialists reveals aligned fine-tuning updates in the middle layers and divergence in the outer layers. This points to a largely language-agnostic reasoning core surrounded by language-specific layers. Exploiting this structure, we introduce a Layer Swap: transferring the English specialist's stronger reasoning mid-layers into each native specialist, closing most of the native reasoning gap across the five non-English languages while preserving CoT in the target language. We release all models and datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
