神经元激活作为统一视角解释大语言模型链式思考激发算术推理的研究
An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs
AI总结：
本研究提出以神经元激活为统一视角，通过分析Llama2前馈层中与算术推理相关的神经元，并借助GPT-4自动识别方法，解释了链式思考提示各组成部分对LLM推理能力的重要性。
摘要：
大语言模型（LLMs）在使用链式思考（Chain-of-Thought，CoT）提示时展现出强大的算术推理能力。然而，我们对LLMs如何处理这些提示的理解仍然有限。为了揭开这一谜团，先前的工作主要集中于消融CoT提示中的不同组成部分，并通过实验观察由此导致的LLM性能变化。但是，这些组成部分为何对LLM推理重要，其原因尚未被探索。为了填补这一空白，本研究以“神经元激活”作为视角，为先前工作所观察到的现象提供统一的解释。具体而言，我们以Llama2为例，考察LLMs前馈层中可能激活其算术推理能力的神经元。为便于研究，我们还提出了一种基于GPT-4的方法，用于自动识别蕴含算术推理的神经元。我们的分析表明，LLM前馈层中推理神经元的激活可以解释CoT提示中各个组成部分的重要性，未来的研究可以在此基础上扩展，以获得更完整的理解。
英文摘要：
Large language models (LLMs) have shown strong arithmetic reasoning capabilities when prompted with Chain-of-Thought (CoT) prompts. However, we have only a limited understanding of how they are processed by LLMs. To demystify it, prior work has primarily focused on ablating different components in the CoT prompt and empirically observing their resulting LLM performance change. Yet, the reason why these components are important to LLM reasoning is not explored. To fill this gap, in this work, we investigate ``neuron activation'' as a lens to provide a unified explanation to observations made by prior work. Specifically, we look into neurons within the feed-forward layers of LLMs that may have activated their arithmetic reasoning capabilities, using Llama2 as an example. To facilitate this investigation, we also propose an approach based on GPT-4 to automatically identify neurons that imply arithmetic reasoning. Our analyses revealed that the activation of reasoning neurons in the feed-forward layers of an LLM can explain the importance of various components in a CoT prompt, and future research can extend it for a more complete understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
