大语言模型的致命弱点：改变少量神经元即可摧毁语言能力
The Achilles' Heel of LLMs: How Altering a Handful of Neurons Can Cripple Language Abilities
AI总结：
研究发现大语言模型中存在关键神经元集，破坏这些神经元可导致模型崩溃，揭示了模型鲁棒性和可解释性的重要问题。
摘要：
大语言模型（LLMs）已成为自然语言处理中的基础工具，驱动着广泛的应用和研究。许多研究表明，LLMs与人脑有显著相似之处。最近的神经科学研究发现，人脑中一小部分生物神经元对核心认知功能至关重要，这引发了一个根本性问题：LLMs是否也包含一小部分关键神经元？本文通过提出基于扰动的因果识别关键神经元方法，系统地定位LLMs中的关键神经元。我们的发现揭示了三个关键见解：（1）LLMs包含超稀疏的关键神经元集。破坏这些关键神经元会使拥有110亿神经元的72B参数模型完全崩溃，困惑度增加高达20个数量级；（2）这些关键神经元并非均匀分布，而是集中在外部层，特别是MLP down_proj组件中；（3）当这些关键神经元被破坏时，性能下降呈现出尖锐的相变，而非渐进式下降。通过在不同模型架构和规模上进行综合实验，我们对这些现象及其对LLM鲁棒性和可解释性的影响进行了深入分析。这些发现可以为开发更稳健的模型架构和改进安全关键应用的部署安全性提供指导。我们的代码可在https://github.com/qqqqqqqzx/The-Achilles-Heel-of-LLMs上获得。
英文摘要：
Large Language Models (LLMs) have become foundational tools in natural language processing, powering a wide range of applications and research. Many studies have shown that LLMs share significant similarities with the human brain. Recent neuroscience research has found that a small subset of biological neurons in the human brain are crucial for core cognitive functions, which raises a fundamental question: do LLMs also contain a small subset of critical neurons? In this paper, we investigate this question by proposing a Perturbation-based Causal Identification of Critical Neurons method to systematically locate such critical neurons in LLMs. Our findings reveal three key insights: (1) LLMs contain ultra-sparse critical neuron sets. Disrupting these critical neurons can cause a 72B-parameter model with over 1.1 billion neurons to completely collapse, with perplexity increasing by up to 20 orders of magnitude; (2) These critical neurons are not uniformly distributed, but tend to concentrate in the outer layers, particularly within the MLP down\_proj components; (3) Performance degradation exhibits sharp phase transitions, rather than a gradual decline, when these critical neurons are disrupted. Through comprehensive experiments across diverse model architectures and scales, we provide deeper analysis of these phenomena and their implications for LLM robustness and interpretability. These findings can offer guidance for developing more robust model architectures and improving deployment security in safety-critical applications. Our code is available at https://github.com/qqqqqqqzx/The-Achilles-Heel-of-LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
