NEAT：LLM中概念驱动的神经元归因
NEAT: Concept driven Neuron Attribution in LLMs
AI总结：
本文提出NEAT方法，利用概念向量定位LLM中负责特定概念的概念神经元，将前向传播次数从O(n*m)降至O(n)，并通过聚类优化搜索，在仇恨言论与偏见分析中验证了其有效性和可干预性。
摘要：
定位对最终预测负责的神经元对于打开大语言模型的黑箱并理解其内部机制具有重要意义。以往研究尝试寻找在神经元层面运作的机制，但这些方法无法表示概念，并且所需的计算量仍有进一步优化的空间。本文借助概念向量，提出了一种定位负责表示特定概念的重要神经元的方法，并将这些神经元称为概念神经元。如果神经元数量为n，示例数量为m，与以往工作相比，我们将所需的前向传播次数从O(n*m)减少到仅O(n)，从而优化了相对于以往工作所需的时间和计算量。我们还将该方法与多个基线及以往方法进行了比较，结果表明我们的方法优于大多数方法，并且与最先进的方法相比更为优化。作为消融研究的一部分，我们还尝试通过引入聚类方法来优化概念神经元的搜索。最后，我们应用所提方法找到并关闭这些神经元，分析其在仇恨言论和LLM偏见等部分中的影响，并在印度语境下评估了偏见部分。我们的方法、分析和解释有助于理解神经元层面对更广泛、更接近人类的概念的责任，也为未来在寻找概念神经元并对其进行干预这一方向上的研究奠定了基础。
英文摘要：
Locating neurons that are responsible for final predictions is important for opening the black-box large language models and understanding the inside mechanisms. Previous studies have tried to find mechanisms that operate at the neuron level but these methods fail to represent a concept and there is also scope for further optimization of compute required. In this paper, with the help of concept vectors, we propose a method for locating significant neurons that are responsible for representing certain concepts and term those neurons as concept neurons. If the number of neurons is n and the number of examples is m, we reduce the number of forward passes required from O(n*m) to just O(n) compared to the previous works and hence optimizing the time and computation required over previous works. We also compare our method with several baselines and previous methods and our results demonstrate better performance than most of the methods and are more optimal when compared to the state-of-the-art method. We, as part of our ablation studies, also try to optimize the search for the concept neurons by involving clustering methods. Finally, we apply our methods to find, turn off the neurons that we find, and analyze its implications in parts of hate speech and bias in LLMs, and we also evaluate our bias part in terms of Indian context. Our methodology, analysis and explanations facilitate understating of neuron-level responsibility for more broader and human-like concepts and also lay a path for future research in this direction of finding concept neurons and intervening them.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
