How related are the representations learned by neural language models, translation models, and language tagging tasks? We answer this question by adapting an encoder-decoder transfer learning method from computer vision to investigate the structure among 100 different feature spaces extracted from hidden representations of various networks trained on language tasks. This method reveals a low-dimensional structure where language models and translation models smoothly interpolate between word embeddings, syntactic and semantic tasks, and future word embeddings. We call this low-dimensional structure a language representation embedding because it encodes the relationships between representations needed to process language for a variety of NLP tasks. We find that this representation embedding can predict how well each individual feature space maps to human brain responses to natural language stimuli recorded using fMRI. Additionally, we find that the principal dimension of this structure can be used to create a metric which highlights the brain's natural language processing hierarchy. This suggests that the embedding captures some part of the brain's natural language representation structure.
RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain
Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所)
Inductive biases, pretraining and fine-tuning jointly account for brain responses to speech
Facebook AI Research(Facebook AI研究院);Université de Paris(巴黎大学);CoML, ENS/CNRS/EHESS/INRIA/PSL University(计算语言学实验室,巴黎高等师范学院/法国国家科学研究中心/法国社会科学高等研究院/法国国家信息与自动化研究所/巴黎文理研究大学);CRI, Département Frontières du Vivant et de l’Apprendre, IIFR, Université de Paris(认知研究所,生命与学习前沿系,法国医学研究院,巴黎大学);École normale supérieure, PSL University, CNRS(巴黎高等师范学院,巴黎文理研究大学,法国国家科学研究中心)
Structural Similarities Between Language Models and Neural Response Measurements
Pioneer Centre for AI, Denmark(丹麦先锋人工智能中心);Center for Philosophy of AI, Denmark(丹麦人工智能哲学中心);University of Copenhagen(哥本哈根大学);Princeton University(普林斯顿大学);Technical University of Denmark(丹麦技术大学)
Large Language Models Align with the Human Brain during Creative Thinking
EPFL(瑞士联邦理工学院洛桑);Università della Svizzera italiana (USI)(意大利语区大学(瑞士));Wesleyan University(卫斯理大学);Paris Brain Institute (ICM)(巴黎大脑研究所);Pennsylvania State University(宾夕法尼亚州立大学)