读者熟练度塑造逐层惊异度剖面

中文摘要 AI 辅助

阅读行为不仅随语言输入而变化，也随读者熟练度而变化。在本研究中，我们探讨了来自大型语言模型（LLMs）的惊异度与人类注视行为之间的逐层关系，是否因读者熟练度水平不同以及注视测量方式不同而有所差异。利用MECO L2语料库的眼动追踪数据，我们比较了高词汇熟练度与低词汇熟练度读者在首次注视时长（FPGD）和总注视时长（TGD）上的表现。我们使用预测深度（Predictive Depth）来量化惊异度预测能力在模型各层之间的分布。在12个测试的LLMs中，我们发现词汇熟练度较低的读者在FPGD上倾向于表现出更深的预测深度，而这一差异在TGD上较小。此外，在两个熟练度组中，TGD本身比FPGD表现出更深的预测深度。这些模式表明，预测能力在LLM各层中的集中位置可能与不同注视测量所捕捉的阅读过程的时间性和广度有关，并且这种关系可能随读者熟练度而变化。我们的留一法分析进一步表明，信息丰富的内部层的优势可扩展到未见文本，尽管预测的实际改进有限。总体而言，我们的结果表明，逐层LLM惊异度为理解不同读者群体和不同注视测量方式下的阅读行为变异提供了一个有用的视角。

英文摘要

Reading behaviour varies not only with linguistic input, but also with reader proficiency. In this study, we investigate whether the layer-wise relationship between surprisal from large language models (LLMs) and human gaze behaviour differs across readers with different levels of proficiency and across gaze measures. Using eye-tracking data from the MECO L2 corpus, we compare readers with high and low vocabulary proficiency on first-pass gaze duration (FPGD) and total gaze duration (TGD). We quantify the distribution of the predictive power of surprisal across model layers using Predictive Depth. Across 12 tested LLMs, we find that readers with lower vocabulary proficiency tend to show deeper Predictive Depth for FPGD, while this difference is smaller for TGD. Also, TGD itself shows deeper Predictive Depth than FPGD in both proficiency groups. These patterns suggest that where predictive power is concentrated across LLM layers may be related to the timing and breadth of the reading processes captured by different gaze measures, and that this relationship can vary with reader proficiency. Our leave-one-out analysis further shows that the advantage of informative internal layers extends to unseen texts, although the practical improvements in prediction are limited. Overall, our results show that layer-wise LLM surprisal provides a useful perspective on variation in reading behaviour across both reader groups and gaze measures.

发表机构

• Universitat Pompeu Fabra（庞培法布拉大学）

机构由 AI 辅助整理，请以论文原文为准。