项目自适应CAT数据中的依赖性与差异项目功能检测:一个多水平框架
Dependencies in Item-Adaptive CAT Data and Differential Item Functioning Detection: A Multilevel Framework
AI总结:
提出两水平逻辑模型,通过考虑CAT诱导的结构依赖性来改进DIF检测,模拟表明该模型能更好控制虚假DIF并具有竞争性统计功效。
摘要:
差异项目功能(DIF)检测是计算机化自适应测试(CAT)中一个重要但研究不足的问题。本文提出一个两水平逻辑模型,通过明确考虑CAT诱导的结构依赖性引起的干扰效应,来改进CAT中的DIF检测。首先,我们概念化自适应项目选择通过临时能力估计在考生和项目之间引入系统依赖性,而传统的单水平DIF方法假设观测独立,在CAT设置中可能产生误导性结果。然后,通过数值示例和蒙特卡洛模拟,我们在各种CAT条件下(操纵测试长度、曝光控制、能力估计器、DIF类型和DIF流行率)将提出的两水平模型与竞争的单水平模型进行比较。报告了每个模型在联合模型收敛条件下的项目水平I类错误率和统计功效。我们表明,与单水平模型相比,提出的两水平模型在控制虚假DIF方面有所改进,并具有竞争性功效,特别是在较短测试和较小曝光率的情况下。然而,我们观察到模型收敛性在模拟条件下系统性地变化,突显出在复杂的CAT DIF设置中,推断准确性和收敛可靠性是相互交织的。通过这项研究,我们强调了多水平DIF建模在CAT中的前景,以及未来研究在评估DIF模型时需要联合评估收敛性和推断性能的必要性。
英文摘要:
Differential item functioning (DIF) detection is an important yet understudied problem in computerized adaptive testing (CAT). In this article, we proposed a two-level logistic model to improve DIF detection in CAT by explicitly accounting for nuisance effects arising from CAT-induced structural dependency. First, we conceptualized that adaptive item selection induces systematic dependencies among examinees and items through provisional ability estimates, whereas traditional single-level DIF methods assume independent observations and may yield misleading results in CAT settings. Then, using a numeric example and Monte Carlo simulations, we compared our proposed two-level model with competing single-level models under various CAT conditions, manipulating test length, exposure control, ability estimator, DIF type, and DIF prevalence. Item-level Type-I error and statistical power conditional on joint model convergence were reported for each model. We showed that the proposed two-level model has improved control of spurious DIF and competitive power relative to single-level models, particularly with shorter tests and smaller exposure rates. However, we observed that the model convergence varied systematically across simulated conditions, highlighting that inferential accuracy and convergence reliability are intertwined in complex CAT DIF settings. Through this study, we underscored both the promise of multilevel DIF modeling in CAT and the need for future research to jointly evaluate convergence and inferential performance when assessing DIF models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。