arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.04484cs.CL
收藏

基于眼动的阅读理解细粒度预测

Fine-Grained Prediction of Reading Comprehension from Eye Movements

Faculty of Data and Decision Sciences, Technion - Israel Institute of Technology(以色列理工学院数据与决策科学学院);Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology(麻省理工学院脑与认知科学系)

Omer Shubi, Yoav Meiri, Cfir Avraham Hadar, Yevgeni Berzak

AI总结:

本文提出在单问题层面依据眼动预测阅读理解,构建三个多模态语言模型并与既有模型比较,在两种阅读模式和多种泛化设置下验证眼动信号的有效性。

摘要:

能否从阅读过程中的眼动来评估人类的阅读理解能力?在本研究中,我们利用面向阅读理解行为分析的文本材料上的大规模眼动追踪数据,来回应这一长期存在的问题。我们聚焦于一项细粒度且在很大程度上尚未得到解决的任务:在文章对应的单个问题层面,根据眼动预测阅读理解情况。我们使用三个新的多模态语言模型,以及文献中已有的一系列模型来完成该任务。我们在普通阅读和信息查找两种不同阅读模式下,评估模型向新文本项目、新参与者以及二者同时为新的组合情形泛化的能力。评估结果表明,尽管该任务极具挑战性,但眼动包含了可用于阅读理解细粒度预测的有用信号。代码和数据将公开发布。

英文摘要:

Can human reading comprehension be assessed from eye movements in reading? In this work, we address this longstanding question using large-scale eyetracking data over textual materials that are geared towards behavioral analyses of reading comprehension. We focus on a fine-grained and largely unaddressed task of predicting reading comprehension from eye movements at the level of a single question over a passage. We tackle this task using three new multimodal language models, as well as a battery of prior models from the literature. We evaluate the models' ability to generalize to new textual items, new participants, and the combination of both, in two different reading regimes, ordinary reading and information seeking. The evaluations suggest that although the task is highly challenging, eye movements contain useful signals for fine-grained prediction of reading comprehension. Code and data will be made publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑