arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.25633cs.AI
收藏

大语言模型中的数学问题解决能力是否与评估表现相关?

Is Mathematical Problem-Solving Expertise in Large Language Models Associated with Assessment Performance?

University of Michigan(密歇根大学);The High School Affiliated to Minzu University of China(中央民族大学附属中学)

Liang Zhang, Yu Fu, Xinyi Jin

AI总结:

研究探讨了大语言模型在数学问题解决能力与评估表现之间的关系,发现模型在解决正确问题时评估准确率更高,但步级诊断仍需额外能力支持。

摘要:

大型语言模型(LLMs)越来越多地用于数学教育,不仅作为问题解决者,也作为学习者推理的评估者。然而,更强的数学问题解决能力是否与更强的步级评估表现相关仍不清楚。本研究使用PROCESSBENCH的GSM8K和MATH子集,该基准用于识别数学推理中的最早错误步骤。我们评估了两种基于LLM的数学辅导代理设置,在相同数学问题上进行两项独立任务:解决问题和评估基准提供的解决方案,预测最早错误步骤。结果表明,模型在解决正确问题时评估准确率显著高于解决错误问题时,且在两种模型和数据集上均具有统计显著性关联。同时,评估比直接问题解决更困难,尤其是在存在错误的解决方案上。这些发现表明,数学问题解决能力支持更强的评估表现,但可靠的步级诊断还需要额外能力,如步骤跟踪、监控和精确错误定位。这些结果对设计和评估支持形成评估的AI辅助自适应教学系统(AIS)有影响。

英文摘要:

Large Language Models (LLMs) are increasingly used in math education not only as problem solvers but also as assessors of learners' reasoning. However, it remains unclear whether stronger math problem-solving ability is associated with stronger step-level assessment performance. This study examines that relationship using the GSM8K and MATH subsets of PROCESSBENCH, a human-annotated benchmark for identifying the earliest erroneous step in mathematical reasoning. We evaluate two LLM-based math tutor agent settings, instantiated with GPT-4 and GPT-5, in two independent tasks on the same math problems: solving the original problem and assessing a benchmark-provided solution by predicting the earliest erroneous step. Results show a consistent within-model pattern: assessment accuracy is substantially higher on math problem items the same model solved correctly than on items it solved incorrectly, with statistically significant associations across both models and datasets. At the same time, assessment remains more difficult than direct problem solving, especially on error-present solutions. These findings suggest that math problem-solving expertise supports stronger assessment performance, but reliable step-level diagnosis also requires additional capabilities such as step tracking, monitoring, and precise error localization. The results have implications for the design and evaluation of AI-supported Adaptive Instructional Systems (AISs) for formative assessment in math education.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑