辅助AI的多选题生成模型
AI-Assisted Model for Generating Multiple-Choice Questions
AI总结:
本文提出了一种人机协同生成多选题的模型,通过AI生成原型并经人类修订,再生成多样化题目以提高质量和覆盖率。
摘要:
多选题(MCQ)被广泛应用于各种教育领域和层次。设计良好的MCQ应评估知识在现实情境中的应用。然而,编写足够数量的测试题目是具有挑战性和耗时的,尤其是在自然科学教育中。足够的MCQ数量问题有两个方面:内容覆盖和考试安全。因此,生成测试题目涉及两个任务:创建MCQ原型并将其转换为题目系列。在自动化题目生成中,原型创建与基于模板的方法如认知建模相对应,而题目扩展对应于基于示例的技术。本研究的目标是设计一个以目标为导向的概念模型,实现人与AI协同创建MCQ,以严格制定的质量标准。所得到的三步模型用于创建MCQ原型,将提示分配给多个AI,每个提示的响应由人类进行修订后再设置下一个提示。为了将MCQ原型转换为MCQ系列,开发了一个一步模型,其中同时生成多个新题目。这些题目评估相同的学习成果,但不是原型或彼此的简单改写。基于人类和自动化评估,大约一半的输出MCQ在无需编辑的情况下是可接受的。对最初被拒绝的测试题进行轻微修改,使系列中的MCQ接受度有所提高,并显著改进了MCQ原型。
英文摘要:
Multiple-choice questions (MCQs) are widely used across diverse educational fields and levels. Well-designed MCQs should evaluate knowledge application in real-world situations. However, writing such test items in sufficient numbers is challenging and time-consuming, especially in natural science education. The problem of a sufficient number of MCQs has two aspects: content coverage and exam security. Therefore, generating test items involves two tasks: creating MCQ prototypes and transforming these prototypes into item series. In automated item generation, prototype creation aligns with template-based methods like cognitive modelling, while item expansion corresponds to example-based techniques. The aim of this research was designing the goal-oriented conceptual model of human - AI co-creation of MCQs that should meet strictly formulated quality criteria. The resulting three-step model for creating MCQ prototypes distributed prompts between several AIs, with human revision of responses for each prompt before setting the next one. To transform the MCQ prototype into an MCQ series, a one-step model was developed in which multiple new items are generated simultaneously. These items assess the same learning outcome but are not simple rephrasings of the prototype or of one another. Based on human and automated evaluation, approximately half of the output MCQs were acceptable without editing. Minor corrections of initially rejected test items allowed for a moderate increase in acceptance of MCQs in series and a significant improvement of MCQ-prototypes.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。