发表机构
Faculty of Data and Decision Sciences; Technion - Israel Institute of Technology（数据与决策科学学院; 以色列理工学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究将眼动预测语言能力的方法扩展至自然段落阅读等场景，提出评分去偏差方法，验证其有效性、可靠性及去偏差效果，为相关评估技术提供更扎实的实证基础。
AI 中文摘要
标准语言能力测试依赖词汇、语法、阅读理解测验等语言任务。Berzak等人（2018）提出了一种受认知启发的替代方法，即从阅读时的眼动行为轨迹预测语言能力。本研究将该方法从单句扩展验证至更具自然性的语境段落阅读（针对英语作为第二语言的学习者）、新能力测量指标、预测模型及信息检索式阅读场景，发现该方法在所有评估中均有效。我们进一步解决基于眼动的能力测试的两个关键开放问题：（1）可能反映读者母语与英语接近程度的潜在评分偏差，这可能损害测试效度；（2）该方法的可靠性。研究发现，基于眼动的能力评分确实偏向于语言上更接近英语的母语（L1）使用者。我们提出的评分去偏差方法可有效解决该问题。可靠性分析表明，眼动能力评分比标准语言能力评分更可靠。总体而言，我们的结果为未来基于眼动的语言评估技术强化并拓宽了实证基础。
英文摘要
Standard language proficiency tests rely on linguistic tasks such as vocabulary, grammar and reading comprehension quizzes. An alternative, cognitively motivated approach, introduced in Berzak et al. (2018), proposed instead to predict language proficiency from behavioral traces of eye movements in reading. In this work, we validate and extend this approach from single sentences to more naturalistic reading of contextualized passages in English as a second language, new proficiency measures, prediction models, and reading in an information seeking regime. We find that the approach is effective in all these evaluations. We further address two key open questions on eye movement based proficiency testing: (1) potential scoring biases that reflect the proximity of the reader's native language to English, which may undermine validity, and (2) its reliability. We find that eye movement based proficiency scores are indeed biased towards L1s that are linguistically closer to English. We propose a score debiasing method which effectively remedies this issue. The reliability analyses suggest that eye movement proficiency scores are more reliable than standard language proficiency scores. Overall, our results strengthen and broaden the empirical foundations for future eye movement based language assessment technologies.
CommentsAccepted to EMNLP
