SPIN：在大型语言模型中稀疏化与整合内部神经元以用于文本分类
SPIN: Sparsifying and Integrating Internal Neurons in Large Language Models for Text Classification
AI总结：
针对 LLM 文本分类仅依赖最后一层输出而忽略内部神经元信息的问题，提出与模型无关的 SPIN 框架，通过逐层线性探针选择并整合跨层显著神经元作为特征，显著提升了分类的准确性、效率和可解释性。
摘要：
在大型语言模型（LLM）带来革命性变革的众多任务中，文本分类便是其中之一。然而，当前的文本分类范式仅依赖 LLM 最后一层的输出，内部神经元中包含的丰富信息在很大程度上未被利用。在本研究中，我们提出了 SPIN：一个与模型无关的框架，用于稀疏化并整合 LLM 中间层的内部神经元以进行文本分类。具体而言，SPIN 通过基于线性探针的逐层显著神经元选择来稀疏化内部神经元，避免了来自无关神经元的噪声并确保了效率。随后，跨层的显著神经元被整合起来，作为分类头的多层特征。大量实验结果表明，我们提出的 SPIN 显著提升了文本分类的准确性、效率和可解释性。
英文摘要：
Among the many tasks that Large Language Models (LLMs) have revolutionized is text classification. Current text classification paradigms, however, rely solely on the output of the final layer in the LLM, with the rich information contained in internal neurons largely untapped. In this study, we present SPIN: a model-agnostic framework that sparsifies and integrates internal neurons of intermediate layers of LLMs for text classification. Specifically, SPIN sparsifies internal neurons by linear probing-based salient neuron selection layer by layer, avoiding noise from unrelated neurons and ensuring efficiency. The cross-layer salient neurons are then integrated to serve as multi-layered features for the classification head. Extensive experimental results show our proposed SPIN significantly improves text classification accuracy, efficiency, and interpretability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
