arXiv 2510.143052026-01-24cs.CL

MathMist: 一种用于数学问题解决和推理的平行多语言基准数据集

MathMist: A Parallel Multilingual Benchmark Dataset for Mathematical Problem Solving and Reasoning

BRAC University（布拉克大学）；United International University（国际联合大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Center for Computational & Data Sciences, Independent University, Bangladesh（计算与数据科学中心，独立大学，孟加拉国）；Amazon GenAI（亚马逊生成人工智能）；University of Virginia（弗吉尼亚大学）

Mahbub E Sobhani, Md. Faiyaz Abdullah Sayeedi, Tasnim Mohiuddin, Md Mofijul Islam, Swakkhar Shatabda

备注：: Accepted for publication in Findings of EACL 2026

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MathMist是一个用于评估多语言数学推理能力的平行多语言基准数据集，涵盖多种语言和资源设置，测试了不同模型在零样本、链式思维等范式下的表现。

摘要：

数学推理仍然是大型语言模型（LLMs）最具挑战性的领域之一，需要不仅有语言理解，还需要结构化的逻辑推断和数值精度。尽管最近的LLMs展示了强大的通用推理能力，但它们在多种语言中的数学能力仍缺乏深入研究。现有的基准测试主要集中在英语或少数高资源语言上，这在评估多语言和跨语言数学推理方面存在显著空白。为此，我们引入了MATHMIST，一种平行多语言基准测试，用于数学问题解决和推理。MATHMIST包含2,890个平行的孟加拉语-英语黄金标准 artifacts，总计约30,000个对齐的问题-答案对，覆盖十三种语言，代表了高、中、低资源语言设置的广泛覆盖。该数据集捕捉了语言多样性、多种问题设置和解决方案综合能力。我们系统地评估了一套多样化的模型，包括开源的小型和中型LLMs、专有系统以及专注于多语言推理的模型，在零样本、链式思维（CoT）、扰动推理和代码切换推理范式下。我们的结果揭示了LLMs在跨语言进行一致和可解释的数学推理方面的持续缺陷，特别是在低资源设置中表现尤为突出。所有代码和数据均在GitHub上可用：https://github.com/mahbubhimel/MathMist

英文摘要：

Mathematical reasoning remains one of the most challenging domains for large language models (LLMs), requiring not only linguistic understanding but also structured logical deduction and numerical precision. While recent LLMs demonstrate strong general-purpose reasoning abilities, their mathematical competence across diverse languages remains underexplored. Existing benchmarks primarily focus on English or a narrow subset of high-resource languages, leaving significant gaps in assessing multilingual and cross-lingual mathematical reasoning. To address this, we introduce MATHMIST, a parallel multilingual benchmark for mathematical problem solving and reasoning. MATHMIST encompasses 2,890 parallel Bangla-English gold standard artifacts, totaling approximately 30K aligned question--answer pairs across thirteen languages, representing an extensive coverage of high-, medium-, and low-resource linguistic settings. The dataset captures linguistic variety, multiple types of problem settings, and solution synthesizing capabilities. We systematically evaluate a diverse suite of models, including open-source small and medium LLMs, proprietary systems, and multilingual-reasoning-focused models under zero-shot, chain-of-thought (CoT), perturbated reasoning, and code-switched reasoning paradigms. Our results reveal persistent deficiencies in LLMs' ability to perform consistent and interpretable mathematical reasoning across languages, with pronounced degradation in low-resource settings. All the codes and data are available at GitHub: https://github.com/mahbubhimel/MathMist

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。