arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1904.01548cs.CLcs.LG
收藏

通过从微调的语言模型预测来理解语言引发的EEG数据

Understanding language-elicited EEG data by predicting it from a fine-tuned language model

Carnegie Mellon University(卡内基梅隆大学)

Dan Schwartz, Tom Mitchell

摘要:

脑电图(EEG)记录的大脑活动,而参与者阅读或听语言被广泛使用的认知神经科学和心理语言学社区作为一种工具来研究语言理解。几个时间锁定的刻板脑电反应的文字介绍-统称为事件相关电位(ERP)-被认为是标记的语义或句法过程中发生的理解。然而,在语言流的哪些特征触发反应方面,每个个体ERP的特征仍然存在争议。改善这一特征将使ERP成为研究语言理解的更有用的工具。我们通过微调语言模型来预测ERP,从而朝着更好地理解ERP迈出了一步。这种新的分析方法首次表明,所有的ERP都可以从语言流的嵌入中预测出来。以前的工作只发现两个ERP是可预测的。除了这种分析,我们研究哪些ERP受益于共享参数在联合培训。我们发现,先前在文献中确定为相互相关的两对ERP从联合训练中受益,而其他几对ERP从联合训练中受益,暗示了潜在的关系。这种分析的扩展,进一步研究什么样的信息在模型嵌入涉及到每个ERP有可能阐明人类语言理解的过程。

英文摘要:

Electroencephalography (EEG) recordings of brain activity taken while participants read or listen to language are widely used within the cognitive neuroscience and psycholinguistics communities as a tool to study language comprehension. Several time-locked stereotyped EEG responses to word-presentations -- known collectively as event-related potentials (ERPs) -- are thought to be markers for semantic or syntactic processes that take place during comprehension. However, the characterization of each individual ERP in terms of what features of a stream of language trigger the response remains controversial. Improving this characterization would make ERPs a more useful tool for studying language comprehension. We take a step towards better understanding the ERPs by fine-tuning a language model to predict them. This new approach to analysis shows for the first time that all of the ERPs are predictable from embeddings of a stream of language. Prior work has only found two of the ERPs to be predictable. In addition to this analysis, we examine which ERPs benefit from sharing parameters during joint training. We find that two pairs of ERPs previously identified in the literature as being related to each other benefit from joint training, while several other pairs of ERPs that benefit from joint training are suggestive of potential relationships. Extensions of this analysis that further examine what kinds of information in the model embeddings relate to each ERP have the potential to elucidate the processes involved in human language comprehension.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑