MMATH：一个用于数学推理的多语言基准
MMATH: A Multilingual Benchmark for Mathematical Reasoning
AI总结：
针对多语言复杂推理能力不足的问题，提出MMATH基准（10种语言、374道数学题），发现模型存在跨语言性能差异和脱靶问题，并证明英语推理加目标语言回答的策略可提升性能与一致性。
摘要：
大型推理模型（如OpenAI o1和DeepSeek R1）的出现显著推进了复杂推理任务的发展。然而，它们在多语言复杂推理方面的能力仍未得到充分探索，现有工作大多聚焦于较简单的任务（如MGSM）。为填补这一空白，我们引入了MMATH，一个覆盖10种类型多样语言、包含374个高质量数学问题的多语言复杂推理基准。使用MMATH，我们观察到即使像DeepSeek R1这样的先进模型在不同语言间也表现出显著的性能差异，并存在一个关键的脱靶问题——生成非预期语言的回答。为解决此问题，我们探索了包括提示和训练在内的策略，证明用英语推理并以目标语言回答可以同时提升性能并保持目标语言一致性。我们的发现为提升大语言模型的多语言推理能力提供了新的见解和实用策略。我们的代码和数据可在https://github.com/RUCAIBox/MMATH获取。
英文摘要：
The advent of large reasoning models, such as OpenAI o1 and DeepSeek R1, has significantly advanced complex reasoning tasks. However, their capabilities in multilingual complex reasoning remain underexplored, with existing efforts largely focused on simpler tasks like MGSM. To address this gap, we introduce MMATH, a benchmark for multilingual complex reasoning spanning 374 high-quality math problems across 10 typologically diverse languages. Using MMATH, we observe that even advanced models like DeepSeek R1 exhibit substantial performance disparities across languages and suffer from a critical off-target issue-generating responses in unintended languages. To address this, we explore strategies including prompting and training, demonstrating that reasoning in English and answering in target languages can simultaneously enhance performance and preserve target-language consistency. Our findings offer new insights and practical strategies for advancing the multilingual reasoning capabilities of large language models. Our code and data could be found at https://github.com/RUCAIBox/MMATH.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
