StructLens:通过最大生成树实现语言模型的结构镜像
StructLens: A Structural Lens for Language Models via Maximum Spanning Trees
AI总结:
本文提出StructLens框架,通过最大生成树分析语言模型的表示结构,揭示模型在不同层和训练阶段中如何组织token表示。
摘要:
语言具有内在结构,这一特性解释了语言习得和语言变化。鉴于此特性,我们预期语言模型也会表现出自身的内部结构。尽管可解释性研究已经探讨了模型如何通过注意力模式和稀疏自编码器计算表示,但所得到的表示的组织方式却被忽视。为解决这一差距,我们引入StructLens,一个通过整体结构视角分析表示的框架。StructLens基于残差流中的语义表示构建最大生成树,受依赖解析中树表示的启发,并在表示空间中提供token关系的摘要。我们分析了连续token在表示空间中也彼此接近,并发现中间层显示出最强的局部跨度组织。此外,对预训练检查点的分析表明,较小的局部单元在预训练早期变得可检测,而较大的单元则在后期才变得可检测。我们的发现表明,StructLens提供了关于模型在不同层和训练过程中如何组织token表示的见解。我们的代码可在https://github.com/naist-nlp/structlens获取。
英文摘要:
Language exhibits inherent structures, a property that explains both language acquisition and language change. Given this characteristic, we expect language models to manifest their own internal structures as well. While interpretability research has investigated how models compute representations mechanistically through attention patterns and Sparse AutoEncoders, the organization of the resulting representations is overlooked. To address this gap, we introduce StructLens, a framework to analyze representations through a holistic structural view. StructLens constructs maximum spanning trees based on the semantic representations in residual streams, inspired by tree representation in dependency parsing, and provides summaries of token relationships in representation space. We analyze how contiguous tokens are also nearby in representation space and find that middle layers show the strongest local-span organization. Moreover, analysis of pre-training checkpoints reveals that smaller local units become detectable earlier in pre-training, and larger units later. Our findings demonstrate that StructLens provides insights into how models organize token representations across layers and training. Our code is available at https://github.com/naist-nlp/structlens.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。