语言作为潜在变量用于推理优化
Language as a Latent Variable for Reasoning Optimization
AI总结：
研究探讨语言作为潜在变量对推理性能的影响，提出polyGRPO框架通过多语言数据优化模型，提升推理准确率和跨任务泛化能力。
摘要：
随着LLM减少英语偏见，非英语响应在推理任务中有时表现更优。本文假设语言作为潜在变量调节模型内部推理路径，而非仅作为输出媒介。通过Polyglot Thinking Experiment验证，非英语响应常更准确，最佳表现出现在语言无约束时，表明多语言性拓宽了模型的潜在推理空间。基于此，提出polyGRPO框架，将语言变化视为隐式探索信号，生成多语言偏好数据，优化策略以提升答案准确性和推理结构。在仅18.1K多语言数学问题上训练，polyGRPO使Qwen2.5-7B-Instruct在四个英语推理测试集上提升6.72%准确率，在多语言基准上提升6.89%。值得注意的是，它是唯一在仅数学数据训练下超越基础LLM的模型，在英语常识推理任务中提升4.9%，显示其强大的跨任务泛化能力。进一步分析表明，将语言视为潜在变量扩展了模型的潜在推理空间，带来一致且可泛化的推理性能提升。
英文摘要：
As LLMs reduce English-centric bias, a surprising trend emerges: non-English responses sometimes outperform English on reasoning tasks. We hypothesize that language functions as a latent variable that structurally modulates the model's internal inference pathways, rather than merely serving as an output medium. To test this, we conducted a Polyglot Thinking Experiment, in which models were prompted to solve identical problems under language-constrained and language-unconstrained conditions. Results show that non-English responses often achieve higher accuracy, and the best performance frequently occur when language is unconstrained, suggesting that multilinguality broadens the model's latent reasoning space. Based on this insight, we propose polyGRPO (Polyglot Group Relative Policy Optimization), an RL framework that treats language variation as an implicit exploration signal. It generates polyglot preference data online under language-constrained and unconstrained conditions, optimizing the policy with respect to both answer accuracy and reasoning structure. Trained on only 18.1K multilingual math problems without chain-of-thought annotations, polyGRPO improves the base model (Qwen2.5-7B-Instruct) by 6.72% absolute accuracy on four English reasoning testset and 6.89% in their multilingual benchmark. Remarkably, it is the only method that surpasses the base LLM on English commonsense reasoning task (4.9%), despite being trained solely on math data-highlighting its strong cross-task generalization. Further analysis reveals that treating language as a latent variable expands the model's latent reasoning space, yielding consistent and generalizable improvements in reasoning performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
