解释量化对大语言模型的影响
Interpreting the Effects of Quantization on LLMs
AI总结：
本研究通过分析不同量化级别对大语言模型内部表示和神经元行为的影响，发现量化对模型校准影响较小，但对神经元冗余的影响因模型而异，表明量化作为压缩技术的有效性需根据具体任务和模型进行评估。
摘要：
量化提供了一种在资源受限环境中部署大语言模型（LLMs）的实用解决方案。然而，其对内部表示的影响仍鲜有研究，引发了关于量化模型可靠性的疑问。在本研究中，我们采用多种可解释性技术，探讨量化如何影响模型和神经元行为。我们分析了多个LLMs在4位和8位量化下的表现。我们的发现表明，量化对模型校准的影响总体上较小。分析神经元激活情况表明，死神经元的数量，即在数据集中激活值接近0的神经元数量，无论量化如何保持一致。在神经元对预测的贡献方面，我们发现较小的全精度模型表现出较少显著的神经元，而较大的模型则倾向于有更多的神经元，但例外的是Llama-2-7B。量化对神经元冗余的影响因模型而异。总体而言，我们的发现表明，量化的影响可能因模型和任务而异，然而我们并未观察到任何剧烈变化，这可能使量化不被视为可靠的模型压缩技术。
英文摘要：
Quantization offers a practical solution to deploy LLMs in resource-constraint environments. However, its impact on internal representations remains understudied, raising questions about the reliability of quantized models. In this study, we employ a range of interpretability techniques to investigate how quantization affects model and neuron behavior. We analyze multiple LLMs under 4-bit and 8-bit quantization. Our findings reveal that the impact of quantization on model calibration is generally minor. Analysis of neuron activations indicates that the number of dead neurons, i.e., those with activation values close to 0 across the dataset, remains consistent regardless of quantization. In terms of neuron contribution to predictions, we observe that smaller full precision models exhibit fewer salient neurons, whereas larger models tend to have more, with the exception of Llama-2-7B. The effect of quantization on neuron redundancy varies across models. Overall, our findings suggest that effect of quantization may vary by model and tasks, however, we did not observe any drastic change which may discourage the use of quantization as a reliable model compression technique.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
