arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.16291cs.LGcs.CL
收藏

可观测传播:揭示Transformer中的特征向量

Observable Propagation: Uncovering Feature Vectors in Transformers

Yale University(耶鲁大学)

Jacob Dunefsky, Arman Cohan

AI总结:

提出一种几乎无需数据的“可观测传播”方法,通过引入“可观测项”概念与耦合系数分析特征向量,在低数据场景下优于传统方法,有效揭示并理解大语言模型中的偏见机制。

摘要:

当前NLP中机制可解释性研究的一个关键目标是为transformer寻找线性特征(也称为“特征向量”):即激活空间中对应于给定模型在其计算中所使用概念的方向。目前寻找线性特征的最先进方法需要大量标注数据——这不仅获取费力,而且利用起来计算成本高昂。在这项工作中,我们引入了一种名为“可观测传播”(简称:ObProp)的新方法,用于寻找transformer语言模型在计算给定任务时所使用的线性特征——几乎不需要数据。我们的范式以“可观测项”的概念为中心,即对应于给定任务的线性泛函。随后,我们引入了用于分析特征向量的数学理论,包括一种称为耦合系数的特征向量间相似度度量,用于估计一个特征的输出与另一个特征输出的相关程度。我们使用ObProp对多个任务进行了广泛的定性研究,包括性别职业偏见、政党预测和编程语言检测。结果表明,在低数据情况下,ObProp超越了寻找特征向量的传统方法,并且ObProp可用于更好地理解大型语言模型中导致偏见的机制。

英文摘要:

A key goal of current mechanistic interpretability research in NLP is to find linear features (also called "feature vectors") for transformers: directions in activation space corresponding to concepts that are used by a given model in its computation. Present state-of-the-art methods for finding linear features require large amounts of labelled data -- both laborious to acquire and computationally expensive to utilize. In this work, we introduce a novel method, called "observable propagation" (in short: ObProp), for finding linear features used by transformer language models in computing a given task -- using almost no data. Our paradigm centers on the concept of "observables", linear functionals corresponding to given tasks. We then introduce a mathematical theory for the analysis of feature vectors, including a similarity metric between feature vectors called the coupling coefficient which estimates the degree to which one feature's output correlates with another's. We use ObProp to perform extensive qualitative investigations into several tasks, including gendered occupational bias, political party prediction, and programming language detection. Our results suggest that ObProp surpasses traditional approaches for finding feature vectors in the low-data regime, and that ObProp can be used to better understand the mechanisms responsible for bias in large language models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑