arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18217cs.CL
收藏

句法理解中的信息论存储成本

Information-Theoretic Storage Cost in Sentence Comprehension

Department of Linguistics, Georgetown University, USA(地缘政治学系,乔治城大学,美国);National Institute for Japanese Language and Linguistics, Japan(日本语言学研究院,日本)

Kohei Kajikawa, Shinnosuke Isono, Ethan Gotlieb Wilcox

AI总结:

本文提出了一种基于信息论的存储成本度量方法,用于评估句法理解过程中上下文信息的存储需求,通过神经语言模型估计该成本,并在英语中验证了其在中心嵌套和相对从句中的处理不对称性,以及在阅读时间变异预测中的有效性。

摘要:

实时句法理解对工作记忆施加了显著负担,因为理解者必须维护上下文信息以预测未来输入。尽管对这种负担的测量在心理语言学理论中起到了重要作用,但它们主要通过符号语法形式化,将句法预测分配为离散且均匀的成本。本研究提出了一种基于信息论形式化的处理存储成本度量,作为先前词语对未来上下文信息的携带量,在不确定性下的度量。与之前的离散、基于语法的度量不同,这种度量是连续的、概率性的、理论中立的,并且可以从预训练的神经语言模型中估计。通过三种英语分析验证了该方法的有效性:我们的度量(i)恢复了已知的中心嵌套和相对从句中的处理不对称性,(ii)与一个语法标注语料库中的基于语法的存储成本相关联,(iii)在两个大规模自然主义数据集中预测阅读时间变异,这在传统信息基础预测器之上。我们的代码可在https://github.com/kohei-kaji/info-storage获取。

英文摘要:

Real-time sentence comprehension imposes a significant load on working memory, as comprehenders must maintain contextual information to anticipate future input. While measures of such load have played an important role in psycholinguistic theories, they have largely been formalized using symbolic grammars, which assign discrete, uniform costs to syntactic predictions. This study proposes a measure of processing storage cost based on an information-theoretic formalization, as the amount of information previous words carry about future context, under uncertainty. Unlike previous discrete, grammar-based metrics, this measure is continuous, probabilistic, theory-neutral, and can be estimated from pre-trained neural language models. The validity of this approach is demonstrated through three analyses in English: our measure (i) recovers well-known processing asymmetries in center embeddings and relative clauses, (ii) correlates with a grammar-based storage cost in a syntactically-annotated corpus, and (iii) predicts reading-time variance in two large-scale naturalistic datasets over and above baseline models with traditional information-based predictors. Our code is available at https://github.com/kohei-kaji/info-storage.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑