R2DE:估计新生成问题的IRT参数的NLP方法
R2DE: a NLP approach to estimating IRT parameters of newly generated questions
摘要:
考试的主要目的是评估学生在某一特定科目上的专业知识。这种专门知识,也称为技能或知识水平,然后可以以不同的方式加以利用(例如,给学生打分,了解学生是否需要一些支持等)。同样,考试中出现的问题在被用来评估学生之前必须以某种方式进行评估。问题评估的标准方法要么是主观的(例如,人类专家的评估)或在问题生成过程中引入长时间延迟(例如,与真正的学生进行预测试)。在这项工作中,我们介绍R2DE(这是一个回归的难度和歧视估计),一个模型,能够评估新生成的多项选择题,通过查看文本的问题和文本的可能的选择。特别是,它可以估计每个问题的难度和区分度,因为它们在项目反应理论中定义。我们还介绍了我们在来自电子学习平台的真实世界大规模数据集上进行的广泛实验的结果,表明我们的模型可用于对新创建的问题进行初步评估,并缓解问题生成中出现的一些问题。
英文摘要:
The main objective of exams consists in performing an assessment of students' expertise on a specific subject. Such expertise, also referred to as skill or knowledge level, can then be leveraged in different ways (e.g., to assign a grade to the students, to understand whether a student might need some support, etc.). Similarly, the questions appearing in the exams have to be assessed in some way before being used to evaluate students. Standard approaches to questions' assessment are either subjective (e.g., assessment by human experts) or introduce a long delay in the process of question generation (e.g., pretesting with real students). In this work we introduce R2DE (which is a Regressor for Difficulty and Discrimination Estimation), a model capable of assessing newly generated multiple-choice questions by looking at the text of the question and the text of the possible choices. In particular, it can estimate the difficulty and the discrimination of each question, as they are defined in Item Response Theory. We also present the results of extensive experiments we carried out on a real world large scale dataset coming from an e-learning platform, showing that our model can be used to perform an initial assessment of newly created questions and ease some of the problems that arise in question generation.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。