QG-SMS：通过学生建模与模拟增强试题分析
QG-SMS: Enhancing Test Item Analysis via Student Modeling and Simulation
AI总结：
针对现有问题生成评估缺乏教育价值关联的不足，本文提出QG-SMS框架，利用大语言模型进行学生建模与模拟，以增强试题分析的有效性和稳健性。
摘要：
虽然问题生成（QG）任务在教育评估中得到了越来越多的应用，但其评估方法仍然受限，缺乏与试题教育价值的明确联系。在本工作中，我们将试题分析（一种教育工作者常用于评估试题质量的方法）引入QG评估。具体而言，我们构建了在主题覆盖度、试题难度、试题区分度和干扰项效率等维度上质量不同的候选问题对。然后，我们检验现有的QG评估方法能否有效区分这些差异。我们的研究结果揭示了这些方法在准确评估与学生学习表现相关的试题质量方面存在显著不足。为弥补这一空白，我们提出了一种新颖的QG评估框架QG-SMS，该框架利用大语言模型进行学生建模与模拟，以执行试题分析。正如我们的大量实验和人工评估研究所展示的，模拟学生档案所引入的额外视角带来了对试题更有效且更稳健的评估。
英文摘要：
While the Question Generation (QG) task has been increasingly adopted in educational assessments, its evaluation remains limited by approaches that lack a clear connection to the educational values of test items. In this work, we introduce test item analysis, a method frequently used by educators to assess test question quality, into QG evaluation. Specifically, we construct pairs of candidate questions that differ in quality across dimensions such as topic coverage, item difficulty, item discrimination, and distractor efficiency. We then examine whether existing QG evaluation approaches can effectively distinguish these differences. Our findings reveal significant shortcomings in these approaches with respect to accurately assessing test item quality in relation to student performance. To address this gap, we propose a novel QG evaluation framework, QG-SMS, which leverages Large Language Model for Student Modeling and Simulation to perform test item analysis. As demonstrated in our extensive experiments and human evaluation study, the additional perspectives introduced by the simulated student profiles lead to a more effective and robust assessment of test items.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
