利用学习进程指导AI反馈促进科学学习
Using Learning Progressions to Guide AI Feedback for Science Learning
AI总结:
本文探讨了基于学习进程的反馈生成管道能否在质量上媲美专家设计的评分标准。研究分析了207名中学生在化学任务中生成的科学解释的AI反馈,发现基于学习进程的评分标准管道在多个维度上与专家设计的评分标准无显著差异。
摘要:
生成式人工智能(AI)提供了可扩展的形成性反馈支持,但大多数AI生成的反馈依赖于由领域专家编写的任务特定评分标准。虽然有效,但评分标准的编写是耗时的,并且限制了在教学情境中的可扩展性。学习进程(LP)提供了一种理论上扎根的学生发展理解的表示,可能提供了一种替代方案。本研究探讨了基于学习进程的评分标准生成管道能否产生与专家设计的评分标准引导的反馈质量相当的AI生成反馈。我们分析了207名中学生在化学任务中生成的科学解释的AI反馈。比较了两种管道:(a)由人类专家设计的任务特定评分标准引导的反馈,和(b)在评分和反馈生成之前从学习进程自动推导出的任务特定评分标准引导的反馈。两位人类编码器使用多维评分标准评估反馈质量,评估清晰度、准确性、相关性、参与度和动机,以及反思性(10个子维度)。评分者间可靠性很高,百分比一致性范围从89%到100%,可估计的维度的Cohen's kappa值(kappa = .66到.88)。配对t检验显示,两种管道在清晰度(t1 = 0.00,p1 = 1.000;t2 = 0.84,p2 = .399)、相关性(t1 = 0.28,p1 = .782;t2 = -0.58,p2 = .565)、参与度和动机(t1 = 0.50,p1 = .618;t2 = -0.58,p2 = .565)或反思性(t = -0.45,p = .656)方面没有统计学上显著差异。这些发现表明,基于学习进程的评分标准管道可以作为一种替代方案。
英文摘要:
Generative artificial intelligence (AI) offers scalable support for formative feedback, yet most AI-generated feedback relies on task-specific rubrics authored by domain experts. While effective, rubric authoring is time-consuming and limits scalability across instructional contexts. Learning progressions (LP) provide a theoretically grounded representation of students' developing understanding and may offer an alternative solution. This study examines whether an LP-driven rubric generation pipeline can produce AI-generated feedback comparable in quality to feedback guided by expert-authored task rubrics. We analyzed AI-generated feedback for written scientific explanations produced by 207 middle school students in a chemistry task. Two pipelines were compared: (a) feedback guided by a human expert-designed, task-specific rubric, and (b) feedback guided by a task-specific rubric automatically derived from a learning progression prior to grading and feedback generation. Two human coders evaluated feedback quality using a multi-dimensional rubric assessing Clarity, Accuracy, Relevance, Engagement and Motivation, and Reflectiveness (10 sub-dimensions). Inter-rater reliability was high, with percent agreement ranging from 89% to 100% and Cohen's kappa values for estimable dimensions (kappa = .66 to .88). Paired t-tests revealed no statistically significant differences between the two pipelines for Clarity (t1 = 0.00, p1 = 1.000; t2 = 0.84, p2 = .399), Relevance (t1 = 0.28, p1 = .782; t2 = -0.58, p2 = .565), Engagement and Motivation (t1 = 0.50, p1 = .618; t2 = -0.58, p2 = .565), or Reflectiveness (t = -0.45, p = .656). These findings suggest that the LP-driven rubric pipeline can serve as an alternative solution.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。