LLMs作为数学考试评分的助教：可靠性与实际可用性
LLMs as Teaching Assistants for Mathematics Exam Grading: Reliability, and Practical Usability
AI总结：
评估六种LLM配置在离散数学考试评分中的表现，发现宽松评分策略降低错误率，但点校准与排名保持仍是不同目标。
摘要：
开放式数学考试很有价值，因为它们评估推理、证明构建、算法思维和中间步骤的沟通。然而，大规模评分困难，因为教师必须一致应用部分评分标准，同时提供帮助学生修复误解的反馈。本文评估了六种当代大型语言模型（LLM）配置——Gemini 3.1 Pro Extended、Gemini 3.5 Flash、ChatGPT 5.5 Pro Extended、ChatGPT 5.5 Thinking、Claude Pro Opus 4.7和Claude Sonnet 4.6——作为本科离散数学考试的评分助手。研究比较了两种评分策略。BASELINE策略使用更严格的遵循评分标准提示，强调明确证据和完整论证。LIBERAL策略是在初步评分显示基线条件有时应用严厉扣分且未能识别有效部分推理后添加的。与人工评分的一致性在题目和考试总分水平上使用平均绝对误差、均方根误差、归一化均方根误差、皮尔逊相关和精确一致度来衡量。结果表明，宽松的部分评分提示降低了每个评估模型族的平均题目级误差。ChatGPT 5.5 Thinking (LIBERAL) 具有最低的平均题目级MAE（1.87）和RMSE（2.53），而Gemini 3.1 Pro Extended (LIBERAL) 具有最低的总分MAE（8.00）和RMSE（10.66）。然而，最强的总分皮尔逊相关出现在Gemini 3.1 Pro Extended (BASELINE) 下，为0.58，表明点校准和排名保持仍然是不同的目标。我们还报告了实际可用性观察。
英文摘要：
Open-ended mathematics exams are valuable because they assess reasoning, proof construction, algorithmic thinking, and communication of intermediate steps. They are also difficult to grade at scale because instructors must apply partial-credit rubrics consistently while giving feedback that helps students repair misconceptions. This paper evaluates six contemporary large language model (LLM) configurations, Gemini 3.1 Pro Extended, Gemini 3.5 Flash, ChatGPT 5.5 Pro Extended, ChatGPT 5.5 Thinking, Claude Pro Opus 4.7, and Claude Sonnet 4.6, as grading assistants for an undergraduate discrete mathematics examination. The study compares two grading policies. The BASELINE policy uses a stricter rubric-following prompt that emphasizes explicit evidence and complete justification. The LIBERAL policy was added after preliminary grading showed that the baseline condition sometimes applied harsh point deductions and failed to recognize valid partial reasoning. Agreement with human grading is measured at both the question and exam-total levels using mean absolute error, root mean squared error, normalized root mean squared error, Pearson correlation, and exact agreement. The results show that liberal partial-credit prompting reduces average question-level error for every evaluated model family. ChatGPT 5.5 Thinking (LIBERAL) has the lowest average question-level MAE (1.87) and RMSE (2.53), while Gemini 3.1 Pro Extended (LIBERAL) has the lowest total-score MAE (8.00) and RMSE (10.66). However, the strongest total-score Pearson correlation occurs under Gemini 3.1 Pro Extended (BASELINE) at 0.58, showing that point calibration and rank preservation remain distinct goals. We also report practical usability observations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
