论人脑与大语言模型的类比：发现语法感知中的关键神经元
On the Analogy between Human Brain and LLMs: Spotting Key Neurons in Grammar Perception
AI总结：
本文利用 Llama 3 识别与词性预测相关的关键神经元，证明大语言模型与人脑类似存在处理语法范畴的专用子空间，并验证其激活模式可可靠预测词性标签。
摘要：
人工神经网络是人工智能的构建模块，其灵感来源于人脑的神经元网络。多年来，这些网络不断演化，以复制大脑的复杂能力，使其能够处理图像和语言处理等任务。在大语言模型领域，人们一直对使语言学习过程更接近人类的方式抱有浓厚兴趣。虽然神经科学研究表明，不同的语法范畴由大脑中不同的神经元处理，但我们证明大语言模型以类似的方式运作。利用 Llama 3，我们识别出与预测属于不同词性标签的单词相关的最重要神经元。利用所获得的知识，我们在一个数据集上训练了一个分类器，结果表明这些关键神经元的激活模式能够可靠地预测新数据上的词性标签。结果提示大语言模型中存在一个专注于捕捉词性标签概念的子空间，这与神经科学中大脑损伤研究观察到的模式相似。
英文摘要：
Artificial Neural Networks, the building blocks of AI, were inspired by the human brain's network of neurons. Over the years, these networks have evolved to replicate the complex capabilities of the brain, allowing them to handle tasks such as image and language processing. In the realm of Large Language Models, there has been a keen interest in making the language learning process more akin to that of humans. While neuroscientific research has shown that different grammatical categories are processed by different neurons in the brain, we show that LLMs operate in a similar way. Utilizing Llama 3, we identify the most important neurons associated with the prediction of words belonging to different part-of-speech tags. Using the achieved knowledge, we train a classifier on a dataset, which shows that the activation patterns of these key neurons can reliably predict part-of-speech tags on fresh data. The results suggest the presence of a subspace in LLMs focused on capturing part-of-speech tag concepts, resembling patterns observed in lesion studies of the brain in neuroscience.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
