AI反馈提升学生修改质量:一项在大规模本科课程中使用FeedbackWriter的随机对照试验
AI-Mediated Feedback Improves Student Revisions: A Randomized Trial with FeedbackWriter in a Large Undergraduate Course
AI总结:
AI反馈系统FeedbackWriter在大规模本科课程中提升学生修改质量,通过随机对照试验发现AI建议能提高修改质量,且TAs采纳越多建议效果越显著。
摘要:
尽管越来越多的人对使用LLMs生成学生写作反馈感兴趣,但很少有人了解学生如何回应AI介导的反馈与人类提供的反馈。我们通过一项随机对照试验填补了这一空白,该试验在一门大规模的入门经济学课程(N=354)中进行,其中我们引入并部署了FeedbackWriter——一个系统,它在教学助理(TAs)为学生撰写的知识密集型论文提供反馈时,生成AI建议。TAs有完全的能力采用、编辑或拒绝这些建议。学生被随机分配接受来自TAs的手写反馈(基线)或AI介导的反馈,其中TAs会收到来自FeedbackWriter的建议。学生根据反馈修改他们的草稿,这些修改随后会被评分。总共使用该系统评阅了1,366篇论文。我们发现,接受AI介导反馈的学生产生了显著高质量的修改,随着TAs采纳更多AI建议,增益增加。TAs发现这些建议有助于发现差距并澄清评分标准。
英文摘要:
Despite growing interest in using LLMs to generate feedback on students' writing, little is known about how students respond to AI-mediated versus human-provided feedback. We address this gap through a randomized controlled trial in a large introductory economics course (N=354), where we introduce and deploy FeedbackWriter - a system that generates AI suggestions to teaching assistants (TAs) while they provide feedback on students' knowledge-intensive essays. TAs have the full capacity to adopt, edit, or dismiss the suggestions. Students were randomly assigned to receive either handwritten feedback from TAs (baseline) or AI-mediated feedback where TAs received suggestions from FeedbackWriter. Students revise their drafts based on the feedback, which is further graded. In total, 1,366 essays were graded using the system. We found that students receiving AI-mediated feedback produced significantly higher-quality revisions, with gains increasing as TAs adopted more AI suggestions. TAs found the AI suggestions useful for spotting gaps and clarifying rubrics.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。