Recent advancements in artificial intelligence have sparked interest in the parallels between large language models (LLMs) and human neural processing, particularly in language comprehension. While prior research has established similarities in the representation of LLMs and the brain, the underlying computational principles that cause this convergence, especially in the context of evolving LLMs, remain elusive. Here, we examined a diverse selection of high-performance LLMs with similar parameter sizes to investigate the factors contributing to their alignment with the brain's language processing mechanisms. We find that as LLMs achieve higher performance on benchmark tasks, they not only become more brain-like as measured by higher performance when predicting neural responses from LLM embeddings, but also their hierarchical feature extraction pathways map more closely onto the brain's while using fewer layers to do the same encoding. We also compare the feature extraction pathways of the LLMs to each other and identify new ways in which high-performing models have converged toward similar hierarchical processing mechanisms. Finally, we show the importance of contextual information in improving model performance and brain similarity. Our findings reveal the converging aspects of language processing in the brain and LLMs and offer new directions for developing models that align more closely with human cognitive processing.
Talking to the brain: Using Large Language Models as Proxies to Model Brain Semantic Representation
South China Normal University(华南师范大学);Center for Studies of Psychological Application, South China Normal University(华南师范大学心理应用研究中心);School of Psychology, South China Normal University(华南师范大学心理学院);Guangdong Key Laboratory of Mental Health and Cognitive Science, South China Normal University(华南师范大学广东省心理健康与认知科学重点实验室)
HIPPD: Brain-Inspired Hierarchical Information Processing for Personality Detection
School of Electronics and Computer Science University of Southampton(电子与计算机科学学院 英国南安普顿大学);Department of Computational Biology Mohamed bin Zayed University of Artificial Intelligence(计算生物学系 摩洛哥本·扎耶德人工智能大学)
BRAINTEASER: Lateral Thinking Puzzles for Large Language Models
Information Sciences Institute, Viterbi School of Engineering, University of Southern California(南加州大学维特比工程学院信息科学研究所);Department of Computer Science, Faculty of Science, Vrije Universiteit Amsterdam(阿姆斯特丹自由大学理学院计算机科学系);Tencent AI Lab(腾讯AI实验室)
PaceLLM: Brain-Inspired Large Language Models for Long-Context Understanding
School of Information Science and Technology, Fudan University(复旦大学信息科学与技术学院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学)
Coherence in the brain unfolds across separable temporal regimes
Department of Adult Psychiatry and Psychotherapy, University of Zurich, Zurich, Switzerland(大学精神病学与心理学系,苏黎世大学,苏黎世,瑞士);Department of Translation and Language Sciences, University Pompeu Fabra, Barcelona, Spain(翻译与语言科学系,庞培法拉大学,巴塞罗那,西班牙);Department of Neuroradiology, Clinical Neuroscience Center, University Hospital Zurich, Zurich, Switzerland(神经放射学系,临床神经科学中心,苏黎世大学医院,苏黎世,瑞士);Department of Psychology, University of Oslo, Oslo, Norway(心理学系,奥斯陆大学,奥斯陆,挪威);Cognitive and Affective Neuroscience Unit, University of Zurich, Zurich, Switzerland(认知与情感神经科学单元,苏黎世大学,苏黎世,瑞士);Center for Clinical Neuroscience and Cognition, University of Groningen, Groningen, Netherlands(临床神经科学与认知中心,格罗宁根大学,格罗宁根,荷兰);Neuroscience Center Zurich, University of Zurich and ETH Zurich, Zurich, Switzerland(苏黎世神经科学中心,苏黎世大学和ETH苏黎世分校,苏黎世,瑞士)
Center for BioMed-X Research, Academy for Advanced Interdisciplinary Studies,Peking University(北京大学生物医学前沿创新中心、前沿交叉学科研究院);Speech and Hearing Research Center, School of Intelligence Science and Technology,Peking University(北京大学智能科学与技术学院言语听觉研究中心);State Key Laboratory of General Artificial Intelligence(通用人工智能国家重点实验室);National Biomedical Imaging Center, State Key Laboratory of Membrane Biology, Institute of Molecular Medicine, Peking-Tsinghua Center for Life Sciences, College of Future Technology, Peking University(国家生物医学成像中心、膜生物学国家重点实验室、分子医学研究所、北京大学生命科学联合中心、未来技术学院)
The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models
Princeton University(普林斯顿大学);Google Research(谷歌研究院);Hebrew University of Jerusalem(希伯来大学);University of California, Los Angeles(加利福尼亚大学洛杉矶分校);New York University Grossman School of Medicine(纽约大学格罗斯曼医学院)
Cross-lingual brain-language model alignment is robust but challenges hierarchical and computational accounts
Grammar and Cognition Lab, Department of Translation & Language Sciences, Universitat Pompeu Fabra(语言与翻译科学系语法与认知实验室,庞培法华大学);Department of Adult Psychiatry and Psychotherapy, University of Zurich(苏黎世大学成人精神病学与心理治疗系);Neuroscience Center Zurich, University of Zurich and ETH Zurich(苏黎世大学神经科学中心与苏黎世联邦理工学院);Center for Clinical Neuroscience and Cognition and Department of Psychiatry, University of Groningen, University Medical Center Groningen(格罗宁根大学临床神经科学与认知中心及精神病学系,格罗宁根大学医学中心);Scalable Scientific Machine Learning Lab, Imperial College London, Department of Earth Science and Engineering(伦敦帝国理工学院可扩展科学机器学习实验室,地球科学与工程系);Institut Català de Recerca i Estudis Avançats (ICREA), Barcelona, Spain(加泰罗尼亚高级研究与研究机构(ICREA),巴塞罗那,西班牙)
Toward a realistic model of speech processing in the brain with self-supervised learning
Ecole Normale Supérieure, PSL University(巴黎高等师范学院,巴黎文理研究大学);Université de Paris cité(巴黎城市大学);Université Paris-Saclay(巴黎萨克雷大学);Inria(法国国家信息与自动化研究所);CEA(法国原子能和替代能源委员会);University of Toronto(多伦多大学);INSERM(法国国家健康与医学研究院);Meta AI
Distinct social-linguistic processing between humans and large audio-language models: Evidence from model-brain alignment
Department of Linguistics and Modern Languages, The Chinese University of Hong Kong(语言学与现代语言系,香港中文大学);Brain and Mind Institute, The Chinese University of Hong Kong(脑与心智研究所,香港中文大学)
Du-IN: Discrete units-guided mask modeling for decoding speech from Intracranial Neural signals
Peking University(北京大学);Chinese Institute for Brain Research(中国脑科学研究院);Beijing Normal University(北京师范大学);Shanghai Jiao Tong University(上海交通大学);Capital Medical University(首都医科大学);Xuanwu Hospital(宣武医院)