PolyMath：评估多语言语境下的数学推理
PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts
AI总结：
本文提出涵盖18种语言和4个难度级别的多语言数学推理基准PolyMath，评估发现先进LLM在多语言推理中存在表现差异大、输入输出语言一致性低及思考长度不同等挑战，并指出控制输出语言可影响推理性能。
摘要：
在本文中，我们介绍了PolyMath，这是一个涵盖18种语言和4个从易到难难度级别的多语言数学推理基准。我们的基准确保了难度的全面性、语言的多样性和高质量的翻译，使其成为推理LLM时代中一个极具区分度的多语言数学基准。我们对高级LLM进行了全面评估，发现即使是Qwen-3-235B-A22B-Thinking和Gemini-2.5-pro，也仅获得了54.6和52.2的基准分数，在最高难度下的准确率约为40%。从语言角度来看，我们的基准揭示了LLM在多语言推理中的几个关键挑战：（1）当前LLM在不同语言间的推理表现差异很大；（2）推理LLM的输入输出语言一致性较低，且可能与性能相关；（3）当前LLM的思考长度因语言而异。此外，我们证明在指令中控制输出语言有可能影响推理性能，特别是对于一些低资源语言，这为提升LLM的多语言能力指明了一个有前景的方向。
英文摘要：
In this paper, we introduce PolyMath, a multilingual mathematical reasoning benchmark covering 18 languages and 4 easy-to-hard difficulty levels. Our benchmark ensures difficulty comprehensiveness, language diversity, and high-quality translation, making it a highly discriminative multilingual mathematical benchmark in the era of reasoning LLMs. We conduct a comprehensive evaluation for advanced LLMs and find that even Qwen-3-235B-A22B-Thinking and Gemini-2.5-pro, achieve only 54.6 and 52.2 benchmark scores, with about 40% accuracy under the highest level From a language perspective, our benchmark reveals several key challenges of LLMs in multilingual reasoning: (1) Reasoning performance varies widely across languages for current LLMs; (2) Input-output language consistency is low in reasoning LLMs and may be correlated with performance; (3) The thinking length differs significantly by language for current LLMs. Additionally, we demonstrate that controlling the output language in the instructions has the potential to affect reasoning performance, especially for some low-resource languages, suggesting a promising direction for improving multilingual capabilities in LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
