arXiv 2502.204082026-01-07q-bio.NCcs.AIcs.CLcs.LG

受脑启发的大型语言模型中功能网络和关键神经元探索

Brain-Inspired Exploration of Functional Networks and Key Neurons in Large Language Models

School of Automation, Northwestern Polytechnical University, Xi’an, China（自动化学院，西北工业大学，西安，中国）；School of Computing, University of Georgia, Athens, USA（计算机学院，佐治亚大学，亚特兰大，美国）；School of Physics and Information Technology, Shaanxi Normal University, Xi’an, China（物理与信息技术学院，陕西师范大学，西安，中国）

Yiheng Liu, Zhengliang Liu, Zihao Wu, Junhao Ning, Haiyang Sun, Sichen Xia, Yang Yang, Xiaohui Gao, Ning Qiang, Bao Ge, Tianming Liu, Junwei Han, Xintao Hu

备注：: 21 pages, 18 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文受脑启发，探索LLM中的功能网络和关键神经元，发现这些网络对模型性能至关重要，通过抑制或增强网络活动可影响模型整体表现或特定任务效果。

摘要：

近年来，大型语言模型（LLMs）在自然语言处理中的快速进步激发了研究人员对其机制和功能特征的兴趣。尽管先前的研究尝试通过识别和解释特定神经元来解释LLM的功能，但这些努力大多集中在单个神经元的贡献上，忽视了人类大脑功能是通过复杂的交互网络实现的。受神经科学领域功能脑网络（FBNs）研究的启发，我们利用在FBN分析中建立的类似方法，探讨LLMs中的

英文摘要：

In recent years, the rapid advancement of large language models (LLMs) in natural language processing has sparked significant interest among researchers to understand their mechanisms and functional characteristics. Although prior studies have attempted to explain LLM functionalities by identifying and interpreting specific neurons, these efforts mostly focus on individual neuron contributions, neglecting the fact that human brain functions are realized through intricate interaction networks. Inspired by research on functional brain networks (FBNs) in the field of neuroscience, we utilize similar methodologies estabilished in FBN analysis to explore the "functional networks" within LLMs in this study. Experimental results highlight that, much like the human brain, LLMs exhibit certain functional networks that recur frequently during their operation. Further investigation reveals that these functional networks are indispensable for LLM performance. Inhibiting key functional networks severely impairs the model's capabilities. Conversely, amplifying the activity of neurons within these networks can enhance either the model's overall performance or its performance on specific tasks. This suggests that these functional networks are strongly associated with either specific tasks or the overall performance of the LLM. Code is available at https://github.com/WhatAboutMyStar/LLM_ACTIVATION.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。