GPT-2的机制可解释性：情感分析中的词汇层与上下文层
Mechanistic Interpretability of GPT-2: Lexical and Contextual Layers in Sentiment Analysis
AI总结：
GPT-2情感分析中，早期层负责词汇检测，晚期层通过统一机制整合上下文现象，推翻了中层专业化假设。
摘要：
我们呈现了一项针对GPT-2的机制可解释性研究，该研究因果地探讨了其Transformer层如何处理情感信息。通过在所有12层上系统地激活修补，我们测试了假设的两阶段情感架构，包括早期的词汇检测和中层的上下文整合。我们的实验确认早期层（0-3）作为词汇情感检测器，编码稳定、位置特定的极性信号，这些信号在很大程度上独立于上下文。然而，所有三个上下文整合假设：中层集中、现象特异性和分布式处理都被推翻。代替中层专业化，我们发现否定、讽刺、领域转换等上下文现象主要通过统一而非模块化的机制在晚期层（8-11）中整合。这些实验发现提供了因果证据，表明GPT-2的情感计算与预测的分层模式不同，突显了进一步经验表征大型语言模型中上下文整合的必要性。
英文摘要：
We present a mechanistic interpretability study of GPT-2 that causally examines how sentiment information is processed across its transformer layers. Using systematic activation patching across all 12 layers, we test the hypothesized two-stage sentiment architecture comprising early lexical detection and mid-layer contextual integration. Our experiments confirm that early layers (0-3) act as lexical sentiment detectors, encoding stable, position specific polarity signals that are largely independent of context. However, all three contextual integration hypotheses: Middle Layer Concentration, Phenomenon Specificity, and Distributed Processing are falsified. Instead of mid-layer specialization, we find that contextual phenomena such as negation, sarcasm, domain shifts etc. are integrated primarily in late layers (8-11) through a unified, non-modular mechanism. These experimental findings provide causal evidence that GPT-2's sentiment computation differs from the predicted hierarchical pattern, highlighting the need for further empirical characterization of contextual integration in large language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
