arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.00034cs.CYcs.AI
收藏

合成学生响应:基于LLM提取的IRT难度参数估计

Synthetic Student Responses: LLM-Extracted Features for IRT Difficulty Parameter Estimation

School of Education, Stanford University(教育学院,斯坦福大学)

Matias Hoyl

AI总结:

利用LLM提取的特征,通过模拟学生响应过程准确估计IRT难度参数,实现高相关性预测。

摘要:

教育评估严重依赖于了解题目难度,传统上通过资源密集的预测试确定。这为课堂教师和评估开发者带来了显著障碍。我们研究是否可以通过建模响应过程来准确估计IRT难度参数,而无需学生测试,并探索不同特征类型对预测准确性的相对贡献。我们的方法结合了传统语言特征和通过大型语言模型(LLM)提取的教育学见解,包括解题步骤数、认知复杂性和潜在的误解。我们实施了两阶段过程:首先训练神经网络预测学生对问题的响应,然后从这些模拟响应模式中推导难度参数。使用超过250,000名学生对数学问题的响应数据集,我们的模型在完全未见过的问题上,预测与实际难度参数之间的皮尔逊相关性约为0.78。

英文摘要:

Educational assessment relies heavily on knowing question difficulty, traditionally determined through resource-intensive pre-testing with students. This creates significant barriers for both classroom teachers and assessment developers. We investigate whether Item Response Theory (IRT) difficulty parameters can be accurately estimated without student testing by modeling the response process and explore the relative contribution of different feature types to prediction accuracy. Our approach combines traditional linguistic features with pedagogical insights extracted using Large Language Models (LLMs), including solution step count, cognitive complexity, and potential misconceptions. We implement a two-stage process: first training a neural network to predict how students would respond to questions, then deriving difficulty parameters from these simulated response patterns. Using a dataset of over 250,000 student responses to mathematics questions, our model achieves a Pearson correlation of approximately 0.78 between predicted and actual difficulty parameters on completely unseen questions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑