评估AI在真实世界中的大学数学手写作业评分：迈向基准的大型研究
Evaluating AI Grading on Real-World Handwritten College Mathematics: A Large-Scale Study Toward a Benchmark
AI总结：
本文提出了一种基于OCR条件的大语言模型，用于评估真实手写数学作业的AI评分，并建立了一个标准化的基准以支持未来研究。
摘要：
在大规模的本科STEM课程中，由于教学工作量大，评分往往无法提供充分的反馈。我们展示了对UC Irvine的实证研究，该研究使用OCR条件下的大语言模型和结构化的评分指南提示，对真实的手写单变量微积分作业进行AI评分。系统为近800名学生的数千份自由回答式测验提交生成了分数和形成性反馈。在没有单一真实标签的环境下，我们评估了系统与官方助教评分、学生调查和独立人类评审的一致性，发现与助教评分高度一致，且大多数AI生成的反馈被评价为正确或可接受。此外，该研究还揭示了OCR条件下数学推理和部分分评估的核心挑战，分析了关键失败模式，提出了实用的评分指南和提示设计原则，并引入了多视角评估协议以确保可靠性和真实课程部署。基于此数据集和评估框架，我们提出了一个标准化的AI手写数学评分基准，以支持可重复的比较和未来研究。
英文摘要：
Grading in large undergraduate STEM courses often yields minimal feedback due to heavy instructional workloads. We present a large-scale empirical study of AI grading on real, handwritten single-variable calculus work from UC Irvine. Using OCR-conditioned large language models with structured, rubric-guided prompting, our system produces scores and formative feedback for thousands of free-response quiz submissions from nearly 800 students. In a setting with no single ground-truth label, we evaluate performance against official teaching-assistant grades, student surveys, and independent human review, finding strong alignment with TA scoring and a large majority of AI-generated feedback rated as correct or acceptable across quizzes. Beyond calculus, this setting highlights core challenges in OCR-conditioned mathematical reasoning and partial-credit assessment. We analyze key failure modes, propose practical rubric- and prompt-design principles, and introduce a multi-perspective evaluation protocol for reliable, real-course deployment. Building on the dataset and evaluation framework developed here, we outline a standardized benchmark for AI grading of handwritten mathematics to support reproducible comparison and future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
