神经语言模型并非生来就适合大脑数据，但训练有助于
Neural Language Models are not Born Equal to Fit Brain Data, but Training Helps
摘要：
神经语言模型（NLM）在过去几年中取得了巨大的进步，在各种语言任务上取得了令人印象深刻的性能。利用这一点，神经科学的研究已经开始使用NLM来研究人类大脑在语言处理过程中的神经活动。然而，关于哪些因素决定了神经语言模型捕捉大脑活动的能力（又称“大脑评分”），仍有许多问题没有答案。在这里，我们在这个方向上迈出了第一步，并检查了测试损失，训练语料库和模型架构（比较GloVe，LSTM，GPT-2和BERT）的影响，对参与者听有声读物的功能磁共振成像时间过程的预测。我们发现：（1）每个模型的未经训练的版本已经通过捕捉相同单词之间大脑反应的相似性来解释大脑中大量的信号，未经训练的LSTM表现优于基于transformer的模型，受上下文影响较小;（2）训练NLP模型提高了相同大脑区域的大脑分数，而不管模型的架构如何;（3）困惑（测试损失）不是大脑评分的良好预测因子;（4）训练数据对结果有很大影响，值得注意的是，现成的模型可能缺乏检测大脑激活的统计能力。总体而言，我们概述了模型训练选择的影响，并为未来旨在使用神经语言模型解释人类语言系统的研究提出了良好的实践。
英文摘要：
Neural Language Models (NLMs) have made tremendous advances during the last years, achieving impressive performance on various linguistic tasks. Capitalizing on this, studies in neuroscience have started to use NLMs to study neural activity in the human brain during language processing. However, many questions remain unanswered regarding which factors determine the ability of a neural language model to capture brain activity (aka its 'brain score'). Here, we make first steps in this direction and examine the impact of test loss, training corpus and model architecture (comparing GloVe, LSTM, GPT-2 and BERT), on the prediction of functional Magnetic Resonance Imaging timecourses of participants listening to an audiobook. We find that (1) untrained versions of each model already explain significant amount of signal in the brain by capturing similarity in brain responses across identical words, with the untrained LSTM outperforming the transformerbased models, being less impacted by the effect of context; (2) that training NLP models improves brain scores in the same brain regions irrespective of the model's architecture; (3) that Perplexity (test loss) is not a good predictor of brain score; (4) that training data have a strong influence on the outcome and, notably, that off-the-shelf models may lack statistical power to detect brain activations. Overall, we outline the impact of modeltraining choices, and suggest good practices for future studies aiming at explaining the human language system using neural language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
