arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.16133cs.AIcs.CLcs.CY
收藏

练习过程中对语言学习的隐性评估与显性测试一样准确

Implicit assessment of language learning during practice as accurate as explicit testing

University of Helsinki(赫尔辛基大学)

Jue Hou, Anisia Katinskaia, Anh-Duc Vu, Roman Yangarber

AI总结:

本研究在智能辅导系统中应用项目反应理论(IRT),证明通过高效适应性测试及直接从练习情境中隐性评估,均能准确估计学习者语言能力,且效果与显性测试相当。

摘要:

学习者能力的评估是智能辅导系统(Intelligent Tutoring Systems, ITS)的重要组成部分。我们在计算机辅助语言学习中使用项目反应理论(Item Response Theory, IRT),在两种情境下评估学生能力:测试环节和练习环节的练习中。对广泛技能进行详尽测试可以提供详细的能力图景,但由于多种原因可能并不理想。因此,我们首先旨在用高效且准确的适应性测试替代详尽测试。我们使用在不完美条件下从详尽测试中收集的学习者数据,训练 IRT 模型来指导适应性测试。模拟和使用真实学习者数据的实验证实了该方法高效且准确。其次,我们探索是否能在不进行测试的情况下,直接从练习情境中准确估计学习者能力。我们将从练习环节收集的学习者数据转换为可用于 IRT 建模的形式。这是通过将练习与语言结构关联来实现的;这些结构随后被视为 IRT 中的“项目”。我们展示了涉及数千名学习者的大规模研究结果。使用教师对学生能力的评估作为“真实标签”,我们比较了从测试与从练习中获得的估计值。实验证实,基于练习,IRT 模型能够产生准确的能力估计。

英文摘要:

Assessment of proficiency of the learner is an essential part of Intelligent Tutoring Systems (ITS). We use Item Response Theory (IRT) in computer-aided language learning for assessment of student ability in two contexts: in test sessions, and in exercises during practice sessions. Exhaustive testing across a wide range of skills can provide a detailed picture of proficiency, but may be undesirable for a number of reasons. Therefore, we first aim to replace exhaustive tests with efficient but accurate adaptive tests. We use learner data collected from exhaustive tests under imperfect conditions, to train an IRT model to guide adaptive tests. Simulations and experiments with real learner data confirm that this approach is efficient and accurate. Second, we explore whether we can accurately estimate learner ability directly from the context of practice with exercises, without testing. We transform learner data collected from exercise sessions into a form that can be used for IRT modeling. This is done by linking the exercises to {\em linguistic constructs}; the constructs are then treated as "items" within IRT. We present results from large-scale studies with thousands of learners. Using teacher assessments of student ability as "ground truth," we compare the estimates obtained from tests vs. those from exercises. The experiments confirm that the IRT models can produce accurate ability estimation based on exercises.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑