混合式教师人工智能评估在学术项目中的应用:效率、公平性与方法论启示
Hybrid Instructor Ai Assessment In Academic Projects: Efficiency, Equity, And Methodological Lessons
AI总结:
本研究提出一种教师监督的生成式AI辅助评估系统,用于水力学报告评分,实现评分时间减少88%、生产力提升733%,且反馈质量与公平性显著提高,符合UNESCO伦理原则。
摘要:
在注册人数众多的技术学科中,例如基础水力学,报告评估需要高水平的客观性、一致性和形成性反馈;这些目标常常因教师工作量而受到影响。本研究介绍了一个由教师监督的生成式人工智能辅助评估系统的实施,用于对33份水力学报告进行评分。核心目标是量化其对流程效率、质量和公平性的影响。所采用的方法论包括使用详细评分标准对大型语言模型进行校准、作业的批量处理,以及人机协同验证阶段。定量结果显示,评分时间显著减少了88%(每份报告从50分钟降至6分钟,包括验证),生产力提高了733%。反馈质量得到大幅提升,表现为100%的评分标准覆盖率和评论锚定到文本证据的比例增加了150%。该系统被证明是公平的,未表现出与报告长度相关的偏见,且校准后高度可靠(分数间r = 0.96)。结论是,混合式人工智能-教师模型优化了评估过程,从而释放了时间用于高价值教学任务,并提高了反馈的公平性和质量,符合联合国教科文组织关于在教育中合乎伦理地使用人工智能的原则。
英文摘要:
In technical subjects characterized by high enrollment, such as Basic Hydraulics, the assessment of reports necessitates superior levels of objectivity, consistency, and formative feedback; goals often compromised by faculty workload. This study presents the implementation of a generative artificial intelligence (AI) assisted assessment system, supervised by instructors, to grade 33 hydraulics reports. The central objective was to quantify its impact on the efficiency, quality, and fairness of the process. The employed methodology included the calibration of the Large Language Model (LLM) with a detailed rubric, the batch processing of assignments, and a human-in-the-loop validation phase. The quantitative results revealed a noteworthy 88% reduction in grading time (from 50 to 6 minutes per report, including verification) and a 733% increase in productivity. The quality of feedback was substantially improved, evidenced by 100% rubric coverage and a 150% increase in the anchoring of comments to textual evidence. The system proved to be equitable, exhibiting no bias related to report length, and highly reliable post-calibration (r = 0.96 between scores). It is concluded that the hybrid AI-instructor model optimizes the assessment process, thereby liberating time for high-value pedagogical tasks and enhancing the fairness and quality of feedback, in alignment with UNESCO's principles on the ethical use of AI in education.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。