Best-of-L：面向数学推理的跨语言奖励建模
Best-of-L: Cross-Lingual Reward Modeling for Mathematical Reasoning
AI总结：
本文提出跨语言奖励模型Best-of-L，通过排序不同语言生成的数学推理回答，证明跨语言采样能显著提升多语言LLM的推理性能，尤其在低采样预算下对英语帮助明显。
摘要：
尽管大语言模型（LLMs）的推理能力持续进步，但在多语言LLMs中，这种能力如何随语言变化，以及不同语言是否会产生相互补充的推理路径，仍不清楚。为研究这一问题，我们训练了一个奖励模型，对给定问题在不同语言下生成的回答进行排序。结果表明，与在单一语言内使用奖励建模相比，我们的跨语言奖励模型显著提升了数学推理性能，甚至使高资源语言也受益。虽然英语在多语言模型中通常表现出最高性能，但我们发现，在低采样预算下，跨语言采样尤其有利于英语。我们的发现揭示了通过利用不同语言的互补优势来改进多语言推理的新机会。
英文摘要：
While the reasoning abilities of large language models (LLMs) continue to advance, it remains unclear how such ability varies across languages in multilingual LLMs and whether different languages produce reasoning paths that complement each other. To investigate this question, we train a reward model to rank generated responses for a given question across languages. Our results show that our cross-lingual reward model substantially improves mathematical reasoning performance compared to using reward modeling within a single language, benefiting even high-resource languages. While English often exhibits the highest performance in multilingual models, we find that cross-lingual sampling particularly benefits English under low sampling budgets. Our findings reveal new opportunities to improve multilingual reasoning by leveraging the complementary strengths of diverse languages.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
