使用深度神经语言模型跟踪自然语言预测
Tracking Naturalistic Linguistic Predictions with Deep Neural Language Models
摘要：
传统上，语言预测的研究是使用简单的设计，其中对预期和意外单词的神经反应以分类的方式进行比较。然而，这些设计一直被认为是“预测错误”，可能夸大了预测在语言理解中的重要性。最近的一些研究已经开始解决这些担忧，使用基于模型的方法来探讨自然主义刺激（例如连续叙事）中语言可预测性的影响。然而，到目前为止，这些研究只关注了非常局部的预测形式，使用的模型在计算单词的可预测性时只考虑了前两个单词。在这里，我们使用最先进的神经语言模型来扩展这种方法，该模型可以考虑大约500倍的语言上下文。从神经网络的可预测性估计提供了一个更好的适合从听自然主义叙事比简单的模型，并揭示了类似于P200和N400的强烈的惊喜反应的受试者的EEG数据。这些结果表明，语言中的可预测性效应不是简单设计的副作用，并展示了人工智能最新进展在语言认知神经科学中的实际应用。
英文摘要：
Prediction in language has traditionally been studied using simple designs in which neural responses to expected and unexpected words are compared in a categorical fashion. However, these designs have been contested as being `prediction encouraging', potentially exaggerating the importance of prediction in language understanding. A few recent studies have begun to address these worries by using model-based approaches to probe the effects of linguistic predictability in naturalistic stimuli (e.g. continuous narrative). However, these studies so far only looked at very local forms of prediction, using models that take no more than the prior two words into account when computing a word's predictability. Here, we extend this approach using a state-of-the-art neural language model that can take roughly 500 times longer linguistic contexts into account. Predictability estimates from the neural network offer a much better fit to EEG data from subjects listening to naturalistic narrative than simpler models, and reveal strong surprise responses akin to the P200 and N400. These results show that predictability effects in language are not a side-effect of simple designs, and demonstrate the practical use of recent advances in AI for the cognitive neuroscience of language.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
