LLaMA模型也有情感：通过探测揭示LLaMA模型中的情感与情绪表示
LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probing
AI总结：
本研究通过探针分类器探测Llama模型的隐藏层，发现情感信息主要集中于中间层，检测准确率较提示技术提升14%，内存需求平均降低57%，为超越提示的情感任务提供了有效方法。
摘要：
大型语言模型（LLMs）已迅速成为NLP的核心，展示了其通过提示技术适应包括情感分析在内的各种任务的能力。然而，我们对这些模型如何捕获情感相关信息仍了解有限。本研究探测了Llama模型的隐藏层，以精确定位情感特征最集中的位置，并评估这如何影响情感分析。使用探针分类器，我们分析了不同层和规模下的情感编码，确定了最能捕获情感信号的层和池化方法。结果表明，对于二元极性任务，情感信息最集中在中间层，检测准确率比提示技术高出14%。此外，我们发现在decoder-only模型中，最后一个token并不总是情感编码信息量最大的。最后，该方法使情感任务得以执行，内存需求平均减少57%。这些见解有助于更广泛地理解LLMs中的情感，表明特定层探测是超越提示的情感任务的有效方法，具有增强模型实用性和降低内存需求的潜力。
英文摘要：
Large Language Models (LLMs) have rapidly become central to NLP, demonstrating their ability to adapt to various tasks through prompting techniques, including sentiment analysis. However, we still have a limited understanding of how these models capture sentiment-related information. This study probes the hidden layers of Llama models to pinpoint where sentiment features are most represented and to assess how this affects sentiment analysis. Using probe classifiers, we analyze sentiment encoding across layers and scales, identifying the layers and pooling methods that best capture sentiment signals. Our results show that sentiment information is most concentrated in mid-layers for binary polarity tasks, with detection accuracy increasing up to 14% over prompting techniques. Additionally, we find that in decoder-only models, the last token is not consistently the most informative for sentiment encoding. Finally, this approach enables sentiment tasks to be performed with memory requirements reduced by an average of 57%. These insights contribute to a broader understanding of sentiment in LLMs, suggesting layer-specific probing as an effective approach for sentiment tasks beyond prompting, with potential to enhance model utility and reduce memory requirements.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
