MathRobust-LV：大型语言模型在数学推理中对语言变体鲁棒性的评估
MathRobust-LV: Evaluation of Large Language Models' Robustness to Linguistic Variations in Mathematical Reasoning
AI总结：
针对数学推理中语言变体鲁棒性评估不足的问题，提出MathRobust-LV测试集，通过改变表面细节保持难度不变，实验发现34个模型准确率普遍下降，揭示推理脆弱性。
摘要：
大型语言模型在数学基准测试中表现出色，但其对语言变体的数学推理鲁棒性尚未得到充分探索。尽管近期研究日益将国际数学奥林匹克（IMO）等高难度竞赛视为评估推理能力的黄金标准，我们仍主张对真实教育场景中的高中水平数学问题进行全面的基准测试。我们提出了MathRobust-LV，一个测试集和评估方法，模拟教师在不同评估中重新表述问题但保持难度不变的方式：我们改变表面细节（如名称、情境、变量），同时保留数值结构和答案。与先前改变问题内容或强调IMO级任务的研究不同，我们聚焦于高中水平数据集问题，其难度级别与当前模型在教育场景（如辅导和评估系统）中的实际部署相匹配。在这些应用中，教师以多种方式重新表述相同概念，使得语言鲁棒性成为可靠部署的关键。尽管MATH数据集的基准测试常被视为已饱和，但我们在34个模型上的实验表明，从基线到变体的准确率有所下降。对于较小的模型，这种下降尤为显著（9-11%），而更强的模型也表现出可测量的性能退化。前沿模型如GPT-5、Gemini-2.5pro则相对稳定。我们的结果强调，对语言变体的鲁棒性是一个基本挑战，揭示了模型在推理方面的脆弱性。
英文摘要：
Large language models excel on math benchmarks, but their math reasoning robustness to linguistic variation is underexplored. While recent work increasingly treats high-difficulty competitions like the IMO as the gold standard for evaluating reasoning, we believe in comprehensive benchmarking of high school-level math problems in real educational settings. We introduce MathRobust-LV, a test set and evaluation methodology that mirrors how instructors rephrase problems across assessments while keeping difficulty constant: we change surface details (names, contexts, variables) while preserving numerical structure and answers. In contrast to prior efforts that alter problem content or emphasize IMO-level tasks, we focus on high-school-level dataset problems at the difficulty level where models are currently deployed in educational settings: tutoring and assessment systems. In these applications, instructors rephrase identical concepts in varied ways, making linguistic robustness essential for reliable deployment. Although MATH data benchmarking is often regarded as saturated, our experiment on 34 models reveals that accuracy declines when moving from the baseline to the variants. These drops are severe for smaller models (9-11%) while stronger models also show measurable degradation. Frontier models like GPT-5, Gemini-2.5pro remain comparatively stable. Our results highlight that robustness to linguistic variation is a fundamental challenge, exposing reasoning vulnerabilities in models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
