人工智能辅助微积分考试手写部分评分研究
Artificial-Intelligence Grading Assistance for Handwritten Components of a Calculus Exam
AI总结：
本研究针对微积分手写题大规模评分需求，验证多模态大模型辅助评分的可行性，设计结合2PL风险度量的人在回路筛选机制，明确性能权衡并提出优化方向，实现常规案例AI处理、特殊案例人工评判的可靠方案。
摘要：
本研究探究当前多模态大语言模型能否在不损害评分有效性的前提下，辅助大规模微积分开放式试题评分工作。在一场大型大一考试中，研究人员采用与助教（TA）一致的评分标准（允许部分得分），使用GPT-5对学生的手写作答进行评分，将助教依据评分标准给出的判定结果作为真值。研究校准了一种人在回路筛选机制，该机制结合部分得分阈值与项目反应理论（2PL）风险度量，风险度量基于AI评分与每个学生-题目对的模型预期得分的偏差计算。未经筛选的AI与助教评分一致性为中等水平，足以支撑低利害反馈场景，但不适用于高利害场景。置信度筛选明确了工作量与评分质量的权衡关系：在更严格的设置下，AI可达到人类水平的评分准确率，但会将约70%的题目留给人工评分。心理测量模式受限于开放式部分权重较低、评分检查点数量较少、指定答题区域与实际作答位置偶尔不匹配等问题。略微提高该部分权重并设置专属答题时间、增设少量评分标准明确的子步骤、强化空间锚定等实际调整措施可提升性能上限。总体而言，校准后的置信度机制与保守路由策略可让AI可靠处理大量常规案例，同时将存在歧义或具备教学价值的作答留给专家评判。
英文摘要：
We investigate whether contemporary multimodal LLMs can assist with grading open-ended calculus at scale without eroding validity. In a large first-year exam, students' handwritten work was graded by GPT-5 against the same rubric used by teaching assistants (TAs), with fractional credit permitted; TA rubric decisions served as ground truth. We calibrated a human-in-the-loop filter that combines a partial-credit threshold with an Item Response Theory (2PL) risk measure based on the deviation between the AI score and the model-expected score for each student-item. Unfiltered AI-TA agreement was moderate, adequate for low-stakes feedback but not for high-stakes use. Confidence filtering made the workload-quality trade-off explicit: under stricter settings, AI delivered human-level accuracy, but also left roughly 70% of the items to be graded by humans. Psychometric patterns were constrained by low stakes on the open-ended portion, a small set of rubric checkpoints, and occasional misalignment between designated answer regions and where work appeared. Practical adjustments such as slightly higher weight and protected time, a few rubric-visible substeps, stronger spatial anchoring should raise ceiling performance. Overall, calibrated confidence and conservative routing enable AI to reliably handle a sizable subset of routine cases while reserving expert judgment for ambiguous or pedagogically rich responses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
