发表机构
Turku Research Institute for Learning Analytics, University of Turku; Department of Mathematics and Statistics, University of Turku（图卢学习分析研究所，图卢大学; 数学与统计系，图卢大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种非参数的基于互信息的框架，通过最大化信息增益确定阈值，以识别大规模评估中的快速反应行为，并在PISA 2022数学数据上验证其可行性。
AI 中文摘要
现有的大规模评估中识别快速反应行为的方法需要对总体进行参数假设。在本研究中，我们提出了一种新颖的、非参数的、基于互信息的框架作为替代方案。该框架内的方法计算观察到的反应和离散化反应时间的互信息，并最大化信息增益以确定区分快速反应与认真参与反应的阈值。这些方法之间的唯一区别在于相关变量的类别数量。我们明确提出了三种方法。第一种方法使用反应正确性和二值化反应时间。第二种方法使用正确性并将时间分为三组。第三种方法使用原始反应和二值化时间。我们将这些方法应用于2022年国际学生评估项目收集的数学成就数据。此外，我们检验了第一种方法在总体水平和实现水平上某些现实条件下的行为和可用性。结果表明，所提出的框架是识别快速反应的一个可行替代方案。该框架的新颖之处在于其非参数性质以及利用原始反应而非正确性的能力。最后，我们讨论了该主题未来可能的研究方向。
英文摘要
Existing methods for identifying rapid-responding behaviour in large-scale assessments require parametric assumptions about the population. In this study, we propose a novel, non-parametric, mutual information-based framework of methods as an alternative. The methods within this framework compute the mutual information of the observed responses and discretised response times and maximise the information gain to determine a threshold that differentiates rapid responses from engaged responses. The only difference between the methods is the number of categories in the relevant variables. We present three methods explicitly. The first method uses response correctness and binarised response times. The second method uses correctness and categorises time into three groups. The third method uses raw responses and binarised times. We applied these methods to mathematics achievement data collected through the Programme for International Student Assessment in 2022. Furthermore, we examined the behaviour and usability of the first method in certain realistic conditions at the population and realised levels. The results indicated that the proposed framework is a viable alternative for identifying rapid responses. The framework's novelty lies in its non-parametric nature and its ability to utilise raw responses instead of correctness. Finally, we discuss some possible future research directions on this topic.
CommentsMain text: 36 pages, 8 figures. Supplement: 12 pages, 1 figure. PISA 2022 data: https://www.oecd.org/en/data/datasets/pisa-2022-database.html. R codes: https://github.com/sajomaho-uni/Empirical-Results-of-MaxMI-for-RRB-Identification. Changes: added references, appendices, better description of AI use, discussion about the advantages of MaxMI, and new analyses
