选择题测试的终结:利用人工智能提升评估效果
The end of multiple choice tests: using AI to enhance assessment
AI总结:
本文提出用AI分析学生对选择题答案的解释,以弥补传统选择题无法揭示学生思维过程的局限,并通过BCI概念验证研究证明该方法能提供可操作反馈、改善学习效果。
摘要:
有效的教学依赖于了解学生知道什么——或自认为知道什么。揭示学生的思维具有挑战性。由于评分简便而常被采用,即使是最好的选择题测试,即那些使用基于研究的干扰项(错误答案)的测试,也因两个因素而在其所能提供的洞察方面存在固有限制。当干扰项不能反映学生的信念时,它们可以被忽略,从而增加正确答案被偶然选中的可能性。此外,做出正确选择并不能保证学生理解其为何正确。为了解决这些局限性,我们建议要求学生解释他们为什么选择自己的答案,以及为什么“错误”的选择是错误的。使用一个经过学科训练的人工智能机器人,可以分析他们的解释,识别可能缺失或被误用的概念和科学原理。该机器人还为教师如何利用这些数据更好地引导学生思维提出建议。在一项小规模“概念验证”研究中,我们使用生物学概念量表(BCI)中的问题测试了这种方法。结果是快速、信息丰富,并提供了关于学生思维的可操作反馈。看来,人工智能的使用解决了传统选择题测试的弱点。将人工智能分析的形成性评估纳入教学,似乎很可能带来整体学习成果的改善。
英文摘要:
Effective teaching relies on knowing what students know-or think they know. Revealing student thinking is challenging. Often used because of their ease of grading, even the best multiple choice (MC) tests, those using research based distractors (wrong answers) are intrinsically limited in the insights they provide due to two factors. When distractors do not reflect student beliefs they can be ignored, increasing the likelihood that the correct answer will be chosen by chance. Moreover, making the correct choice does not guarantee that the student understands why it is correct. To address these limitations, we recommend asking students to explain why they chose their answer, and why "wrong" choices are wrong. Using a discipline-trained artificial intelligence-based bot it is possible to analyze their explanations, identifying the concepts and scientific principles that maybe missing or misapplied. The bot also makes suggestions for how instructors can use these data to better guide student thinking. In a small "proof of concept" study, we tested this approach using questions from the Biology Concepts Instrument (BCI). The result was rapid, informative, and provided actionable feedback on student thinking. It appears that the use of AI addresses the weaknesses of conventional MC test. It seems likely that incorporating AI-analyzed formative assessments will lead to improved overall learning outcomes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。